# Small models for small jobs

Use a small model (like gpt-4o-mini) for routing and simple steps, and a big model only where reasoning matters.

AI developers · Part 09: Production · Card 10 of 13 · Last updated 2026-10-13

## Analogy

You don't spin up a GPU instance to serve a favicon.

## In an agent

The biggest cost lever in most agents. Track cost per request from usage tokens to see it working.

## Practice

### Flashcards

- **Interview: biggest lever for agent cost?** Route simple steps to small models and use big models only for hard reasoning.

### Interview questions

**How do you reduce cost and latency?**

- Small models for routing and simple steps; big models only for hard reasoning.
- Caching and short tool results.
- Parallel tool calls and streaming.
- Track cost per request from usage tokens.

## Questions people ask

### How do I reduce the cost of an AI agent?

Route by difficulty. Use a small, cheap model, such as gpt-4o-mini, for routing, classification and simple steps, and a large model only where real reasoning is needed. Also keep tool results short and cache repeated work. Track cost per request from the usage tokens to confirm each change helped.

### Will a smaller model make my agent worse?

Only if it does a job that needs more reasoning. Small models handle routing, extraction and simple tool calls well. Test each step on your eval set with the small model; keep the large model for the steps where scores really drop. That way you save money without losing quality.

## Related concepts

- [Cost and latency tricks](https://studyvisually.com/learn/tech/ai-developers/production/cost-and-latency)
- [Streaming](https://studyvisually.com/learn/tech/ai-developers/openai-api/streaming)
- [Guardrails](https://studyvisually.com/learn/tech/ai-developers/production/guardrails)
- [Validate tool arguments](https://studyvisually.com/learn/tech/ai-developers/production/validate-tool-arguments)

Web page: https://studyvisually.com/learn/tech/ai-developers/production/model-routing-for-cost
Study this card in the app: https://studyvisually.com/app/learn?category=ai-developers&card=model-routing-for-cost