AI developers · Part 05
Embeddings + RAG
Turn text into vectors, search them, and ground answers in your own data.
Last updated
- EmbeddingA list of numbers (a vector, often 384 or 1536 long) that captures the meaning of a piece of text. Similar meanings get similar numbers.
- Cosine similarityA score of how close two embeddings are in meaning. 1 means the same meaning, 0 means unrelated.
- Semantic searchSearch by meaning instead of exact words. It finds "rollback steps" when you asked about reverting a deploy.
- Vector databaseA database that stores embeddings and finds the nearest ones fast. pgvector (Postgres), MongoDB Atlas Vector Search and Pinecone are common.
- RAGRetrieval-Augmented Generation: search your own data first, paste the best matches into the prompt, then let the model answer from them.
- Indexing (run once)Split your documents into chunks, embed each chunk, and store the chunk text with its vector.
- Querying (every question)Embed the question, fetch the closest chunks, and send them to the model with an instruction to answer only from that context.
- Answer only from contextTell the model to answer only from the retrieved text, and to say "I don't know" when the answer isn't there.
- Chunk sizeStart with 300 to 800 tokens per chunk and about 10% overlap. Split on headings and paragraphs, or functions for code.
- Top KHow many chunks you retrieve per question. Start with 3 to 5.
- MetadataStore the source file, section and date with each chunk.
- Hybrid searchCombine keyword search with vector search.
- Re-rankingA second model re-orders the top 20 results by relevance, then you keep the best few.
- Agentic RAGRetrieval becomes a tool, searchDocs(query). The agent decides when to search, and can search several times with different queries.