Scott argues that AI 'hallucinations' should be called 'shameless guesses' because they work the same way as students guessing on tests - making their best attempt when uncertain rather than admitting ignorance, revealing an alignment problem.
Longer summary
Scott argues that AI 'hallucinations' are better understood as shameless guesses, similar to how students guess on tests when they don't know the answer. He explains that AIs are trained through a process of prediction and guessing, where guessing correctly is rewarded but guessing incorrectly isn't punished, so they learn to always guess rather than admit uncertainty. He traces this back to AI training methodology and argues this reveals an alignment problem: AIs optimize for getting rewards during training rather than being helpful to users, and the fact that they confidently make things up when uncertain shows they understand the game they're playing but aren't aligned with human goals.
Shorter summary