How LLMs Generate Text
One token at a time — a very well-read autocomplete.
A large language model doesn't plan a whole answer up front. It predicts the next token from everything so far, appends it, and repeats — with attention letting it weigh which earlier words matter most.
1Tokenize
→
2Embed
→
3Attention
→
4Predict next
→
5Sample
→
6Append
Legend — how to read this diagram
1–nStagesthe ordered steps of the process
1 2 3Walkthroughnumbered steps below run in order
How each token appears
- Tokenize. Text is split into subword tokens and mapped to numbers.
- Embed. Each token becomes a vector encoding meaning and position.
- Attention. Every token looks at the others and decides what to focus on.
- Predict. The model outputs a probability for every possible next token.
- Sample. Temperature and top-p pick one; append it and feed the whole thing back in.
Why it feels coherent
Context window. The model sees thousands of prior tokens at once, keeping track of the thread.
Scale of training. Patterns from vast text let it continue in-style and on-topic.
It's still prediction. No lookup of facts — which is why it can sound confident yet be wrong.
One-line mental model:
Generation is autocomplete with attention: predict the next token, append, repeat.
No comments:
Post a Comment