# Chunk size

Start with 300 to 800 tokens per chunk and about 10% overlap. Split on headings and paragraphs, or functions for code.

AI developers · Part 05: Embeddings + RAG · Card 9 of 14 · Last updated 2026-10-13

## Analogy

Pagination for documents. Pages too small lose the plot; pages too big bury the one line you needed.

## In an agent

Bad chunks are a top cause of bad answers. Tune chunk size with retrieval evals, not vibes.

## Practice

### Flashcards

- **What is a chunk?** A small piece of a document, stored for retrieval.
- **Interview: how do you pick chunk size?** Start at 500 to 800 tokens with overlap, split on natural boundaries, then tune with retrieval evals.

### Interview questions

**How do you choose chunk size?**

- Start around 500 to 800 tokens with some overlap.
- Split on natural boundaries like headings, paragraphs or functions.
- Tune with retrieval evals instead of guessing.

## Questions people ask

### What chunk size should I use for RAG?

Start with 300 to 800 tokens per chunk and about 10 percent overlap, splitting on headings and paragraphs, or on functions for code. Then tune it with retrieval evals on real questions. Chunks that are too small lose context; chunks that are too big bury the one line you needed.

### Why do bad chunks cause bad RAG answers?

Retrieval returns whole chunks. If a chunk cuts an explanation in half, the model gets half an idea. If it mixes several topics, the useful sentence drowns in noise and may not rank highly at all. Splitting along natural boundaries keeps each chunk about one complete idea.

## Related concepts

- [Text splitters](https://studyvisually.com/learn/tech/ai-developers/langchain-langgraph/text-splitters)
- [Indexing (run once)](https://studyvisually.com/learn/tech/ai-developers/embeddings-rag/rag-indexing)
- [Evals](https://studyvisually.com/learn/tech/ai-developers/production/evals)
- [RAG](https://studyvisually.com/learn/tech/ai-developers/embeddings-rag/rag)

Web page: https://studyvisually.com/learn/tech/ai-developers/embeddings-rag/chunking
Study this card in the app: https://studyvisually.com/app/learn?category=ai-developers&card=chunking