# Cost and latency tricks

Cache repeated answers and embeddings, keep tool results short (5 fields, not the whole API response), run tool calls in parallel, and stream the answer.

AI developers · Part 09: Production · Card 11 of 13 · Last updated 2026-10-13

## Analogy

The usual web perf checklist: cache, trim payloads, parallelise, show something fast.

## In an agent

Every token in a tool result is paid for again on every later loop step.

## Practice

### Flashcards

- **Interview: biggest lever for agent cost?** Route simple steps to small models and use big models only for hard reasoning.

### Interview questions

**How do you reduce cost and latency?**

- Small models for routing and simple steps; big models only for hard reasoning.
- Caching and short tool results.
- Parallel tool calls and streaming.
- Track cost per request from usage tokens.

## Questions people ask

### How do I make an AI agent faster?

Run independent tool calls in parallel, keep tool results short so each model call is smaller, cache repeated answers and embeddings, use small models for simple steps, and stream the final answer so users see progress immediately. It is the usual web performance checklist applied to agents.

### Why should tool results be short?

Every token in a tool result stays in the messages and is paid for again on every later loop step. A full API response can be thousands of tokens when the model needs five fields. Return only those fields, and the agent gets cheaper, faster and less distracted.

## Related concepts

- [Small models for small jobs](https://studyvisually.com/learn/tech/ai-developers/production/model-routing-for-cost)
- [Streaming](https://studyvisually.com/learn/tech/ai-developers/openai-api/streaming)
- [Guardrails](https://studyvisually.com/learn/tech/ai-developers/production/guardrails)
- [Validate tool arguments](https://studyvisually.com/learn/tech/ai-developers/production/validate-tool-arguments)

Web page: https://studyvisually.com/learn/tech/ai-developers/production/cost-and-latency
Study this card in the app: https://studyvisually.com/app/learn?category=ai-developers&card=cost-and-latency