Context window
The maximum number of tokens the model can handle in one call: system prompt, history, tool results and the answer all count.
Last updated
Analogy
A max request body size. Go over it and the call fails or gets cut off, like uploading a 4K video to a form that accepts 2 MB.
Code example
const CONTEXT_LIMIT = 128_000; // gpt-4o-mini const usedTokens = res.usage.prompt_tokens;if (usedTokens > CONTEXT_LIMIT * 0.8) { messages = await summarizeOldTurns(messages);}In an agent
Each loop iteration resends the whole history, so the window fills up fast. You must trim or summarize.
Practice
Flashcards
- What is a context window?The maximum number of tokens a model can handle in one call.
Questions people ask
What happens when a conversation is bigger than the context window?
The call fails, or the oldest text gets cut off, depending on the provider and your settings. Either way the model loses information. Agents hit this fast because every loop step resends the whole history. The fix is to trim old messages or summarise them before the window fills up.
What counts toward the context window?
Everything in one call: the system prompt, every earlier user and assistant message, every tool call and tool result, and the answer the model writes back. Tool results are often the biggest part, so returning a few useful fields instead of a whole API response saves a lot of room.