People love to say an LLM is "just fancy autocorrect." Watch what really happens to a single word — bank — and you'll see it's something else entirely.
{{ captionBody }}
Long or rare words get split too. “deposit” might become de + posit, and “librarian” → librari + an. The model always works in these pieces — that's all a token is.
Autocorrect guesses the next word from raw frequency — every “bank” looks the same to it. Attention does the opposite: it lets the sentence change what a word means. That single idea is what turns a list of numbers into understanding.
See the space these words live in