AI developers · Part 01
LLM basics
Tokens, context windows, temperature, and why models forget everything between calls.
Last updated
- LLMA Large Language Model is a function that takes text in and gives text back. GPT-4o, Claude and Llama are all LLMs.
- Next-token predictionAn LLM writes one small piece of text at a time. Each piece is its best guess of what comes next, given everything before it.
- Who decides the next stepAI in an app comes in four levels: a raw LLM, an LLM integration (your code calls it for one job), a workflow (fixed steps, AI is one of them), and an agent (the model picks the steps).
- TokenA chunk of text, about 4 characters or three quarters of an English word. Models read and write tokens, not letters.
- Context windowThe maximum number of tokens the model can handle in one call: system prompt, history, tool results and the answer all count.
- StatelessThe model remembers nothing between calls. Each request starts from a blank slate.
- Message rolesEach message has a role: system (the rules), user (the human), assistant (the model) and tool (results of a function your code ran).
- System promptStanding instructions that set the model's role, tone and rules for the whole conversation.
- TemperatureA dial for randomness. 0 is focused and repeatable; 1 is more varied and creative.
- HallucinationWhen the model confidently makes something up: a fake function, a wrong date, a library that doesn't exist.