Temperature
A dial for randomness. 0 is focused and repeatable; 1 is more varied and creative.
Last updated
Analogy
Seeding a random generator. Low temperature is the boring colleague who answers the same way every time, which is exactly what you want in prod.
Code example
const res = await client.chat.completions.create({ model: "gpt-4o-mini", temperature: 0, // same input, (almost) same output messages,});In an agent
Keep it low (0 to 0.3) for agents and tool calls so the model picks tools reliably.
Practice
Flashcards
- What does temperature control?How random the output is. Keep it low (0 to 0.3) for agents.
- Interview: how do you reduce hallucinations?Ground with RAG and tools, allow "I don't know", ask for citations, use low temperature, and test with evals.
Interview questions
- How do you handle hallucinations?
- Ground answers with RAG and tools.
- Instruct the model to say "I don't know" when the context lacks the answer, and ask for citations.
- Use a low temperature and test with evals.
Questions people ask
What temperature should I use for an agent?
Keep it low, between 0 and 0.3. Agents need to pick the same tool for the same situation and produce arguments your code can parse. Low temperature makes the output more repeatable and easier to test. Save higher values for creative writing, where variety is the point.
Does temperature 0 make an LLM fully deterministic?
Almost, but not always. At temperature 0 the model picks its most likely next token, so the same input usually gives the same output. Small differences can still appear between runs. Treat it as more repeatable, not guaranteed identical, and test with evals rather than exact string matches.