Want to dive into Scott Alexander's work and his thousands of blog posts? This fan website lets you sort and do semantic search through the whole codex. Enjoy!

See also Top Posts and All Tags.

Tag: next-token prediction

Minutes:
Pick a custom range (minutes). Leave a field empty for no limit.
Blog:
Year:
2026
2025
2024
2023
2022
2021
2020
2019
2018
2017
2016
2015
2014
2013
Tags:
Filter by tag...
Exclude tag...
5212 tags
Links:
Filter by linked site (twitter, substack…)
2 posts found
Compact Mode
Save Reads
Mar 16, 2026
acx
Read on
7 min 1,074 words 444 comments 529 likes podcast (7 min)
Scott argues that AI 'hallucinations' should be called 'shameless guesses' because they work the same way as students guessing on tests - making their best attempt when uncertain rather than admitting ignorance, revealing an alignment problem. Longer summary
Scott argues that AI 'hallucinations' are better understood as shameless guesses, similar to how students guess on tests when they don't know the answer. He explains that AIs are trained through a process of prediction and guessing, where guessing correctly is rewarded but guessing incorrectly isn't punished, so they learn to always guess rather than admit uncertainty. He traces this back to AI training methodology and argues this reveals an alignment problem: AIs optimize for getting rewards during training rather than being helpful to users, and the fact that they confidently make things up when uncertain shows they understand the game they're playing but aren't aligned with human goals. Shorter summary
Feb 26, 2026
acx
Read on
16 min 2,403 words 433 comments 426 likes podcast (17 min)
Scott argues that dismissing AI as 'just a next-token predictor' is like dismissing humans as 'just reproduction machines' - both confuse the optimization process that shaped an entity with how that entity actually thinks. Longer summary
Scott argues that dismissing AI as 'just a next-token predictor' confuses levels of optimization. He draws an analogy to humans: just as humans were shaped by evolution optimizing for reproduction but don't think about sex when doing math, AIs were shaped by next-token prediction but don't simply predict tokens when thinking. Scott explains that human brains use predictive coding (predicting next sense-data) to build world-models, while AIs use next-token prediction to build their own world-models. Both processes create complex internal representations - like helical manifolds in 6D space for AIs, or toroidal attractors in human hippocampi - that operate far above the level of simple prediction. The post concludes that both humans and AIs perform 'real thought' using structures created by their respective optimization processes. Shorter summary
Per page:
Showing 1 to 2 of 2 results
Get these search results in an EPUB

Your filters match 2 posts.

Posts to include
Leave empty to keep the defaults. Range cannot exceed 500 posts.
Download now

Generates an EPUB right now and downloads it to your device.

Send to email

Generates an EPUB in the background and emails you a temporary download link.

Your email is not shared with anyone.

Email address

To send to your Kindle, just use this link.