Local embeddings
Run an embedding model like all-MiniLM-L6-v2 on your own server. It downloads once (tens of MB), then runs on CPU.
Last updated
Analogy
Self-hosting your own image resizer instead of paying per image to a SaaS.
Code example
const embed = await pipeline("feature-extraction", "Xenova/all-MiniLM-L6-v2");const output = await embed("How do I roll back a deploy?", { pooling: "mean", normalize: true,});const vector = Array.from(output.data); // 384 numbersIn an agent
No API key, no per-call cost, and your docs never leave your server.
Practice
Flashcards
- Interview: when do you pick an open-source model?High-volume narrow tasks (embeddings, classification), strict privacy, or when cost per call matters.
Questions people ask
How do I create embeddings locally without an API key?
Use Transformers.js with pipeline feature-extraction and a small model such as Xenova/all-MiniLM-L6-v2. It downloads once, tens of megabytes, then runs on the CPU. Call it with mean pooling and normalisation turned on, and convert the output to a plain array of numbers to store in your vector database.
Are local embeddings good enough for RAG?
For many projects, yes. Small models like all-MiniLM-L6-v2 handle typical English documentation well, cost nothing per call and keep your documents on your own server. Hosted embedding models can rank better on harder or multilingual data. Compare both on your own retrieval evals before deciding.