RAG
Retrieval-Augmented Generation: search your own data first, paste the best matches into the prompt, then let the model answer from them.
Last updated
Analogy
An open-book exam. The model is still the student, but now it gets to bring your docs into the room.
In an agent
RAG fixes two problems at once: the model doesn't know your private data, and it makes things up when it doesn't know.
Practice
Flashcards
- What is RAG?Retrieve relevant chunks from your own data, add them to the prompt, and have the model answer from them.
- Interview: why use RAG?It gives the model private, up-to-date knowledge and reduces hallucination, without fine-tuning.
- Interview: RAG or fine-tuning for company knowledge?RAG. It adds knowledge and updates by re-indexing. Fine-tuning changes behaviour and needs retraining.
Interview questions
- What is RAG and why use it?
- Retrieve relevant chunks from your own data and add them to the prompt.
- The model answers from that context instead of from memory.
- It gives the model private, up-to-date knowledge and reduces hallucination, without fine-tuning.
- RAG vs fine-tuning?
- RAG adds knowledge and is easy to update: re-index a document.
- Fine-tuning changes behaviour or style and needs training data and retraining.
- For company knowledge, choose RAG almost every time.
- How do you handle hallucinations?
- Ground answers with RAG and tools.
- Instruct the model to say "I don't know" when the context lacks the answer, and ask for citations.
- Use a low temperature and test with evals.
Questions people ask
What is RAG in simple terms?
RAG, retrieval-augmented generation, means searching your own data first, pasting the best matches into the prompt, and then letting the model answer from them. It is an open-book exam: the model still writes the answer, but it can read your documents while it does.
What problems does RAG solve?
Two at once. The model does not know your private or recent data, and it makes things up when it does not know. RAG hands it the relevant text at question time, so answers come from your documents. Updating knowledge means re-indexing documents, with no model training needed.