Next-token prediction
An LLM writes one small piece of text at a time. Each piece is its best guess of what comes next, given everything before it.
Last updated
Analogy
Autocomplete in your editor, but it read every repo on GitHub and refuses to stop at one suggestion.
In an agent
The model only reacts to the text in front of it. So an agent is only as good as the messages, tool results and instructions you put there.
Questions people ask
Does an LLM plan its whole answer before writing it?
Not the way a person does. It writes one token at a time, and each token is its best guess given everything before it. That is why clear instructions and good context near the end of the prompt change the answer so much: the model only reacts to the text in front of it.
Why does next-token prediction matter when building agents?
Because the model only reacts to the text you give it. An agent's quality depends on what lands in its messages: the system prompt, the tool results and the history. Feed it vague or noisy text and it predicts vague or noisy steps. Feed it precise context and its next step gets much better.