AI's Worst Failure Mode Isn't Hallucination

From Torvalds' kernel debug to clinical notes that blind patients, the failure modes AI can't catch about itself and why RLHF makes them worse.