After running the vector database comparison for my RAG pipeline, I realized I had been optimizing the wrong thing. The database choice changed my recall numbers by a few percentage points. The embedding model choice changed them by twenty. The model is the retrieval system. The database is the index.
This post covers what I found testing the main self-hostable embedding models on the same retrieval task, using the same vector database throughout so the numbers reflect the model rather than the index.
Why the Embedding Model Matters More Than You Might Expect
An embedding model takes a text chunk and produces a vector. Two chunks that are semantically similar should produce vectors that are close in the embedding space. A query should produce a vector close to the vectors of relevant documents.
The quality of this mapping determines how well retrieval works. A better embedding model finds the right documents even when the query uses different words than the document. A weaker model requires close lexical overlap, which makes it brittle on the kinds of questions real users ask.
The vector database's HNSW index searches through the embedding space efficiently. But if the embedding space itself is poorly organized — if semantically similar documents end up far apart because the model did not learn the right representations — no index strategy fixes that.
The Models I Tested
All five run via Ollama on local hardware without an API dependency:
nomic-embed-text (nomic-ai/nomic-embed-text-v1.5): a 137M-parameter model trained specifically for retrieval. Nomic released it with a permissive license and it has become the default recommendation for local RAG setups. Available at 768 dimensions.
mxbai-embed-large (mixedbread-ai/mxbai-embed-large-v1): a 335M-parameter model from Mixedbread AI, trained on a large diverse corpus with a focus on retrieval quality. Available at 1024 dimensions. The larger size costs more in RAM and indexing time.
bge-m3 (BAAI/bge-m3): from the Beijing Academy of Artificial Intelligence, trained for multilingual retrieval. Supports multiple retrieval modes (dense, sparse, and multi-vector). At 567M parameters, it is the largest of the five.
all-minilm-l6-v2: a 22M-parameter model optimized for speed. A common default in many tutorials and starter projects. Available at 384 dimensions.
snowflake-arctic-embed-m: a 109M-parameter model from Snowflake, trained specifically for retrieval with a focus on asymmetric search (short queries, longer documents).
The Benchmark
I used the same document corpus from the vector database comparison: 500,000 document chunks from a code documentation set, stored in Qdrant. I ran 1,000 query samples with ground-truth relevance labels, measuring:
- Recall@10: did a relevant document appear in the top 10 results?
- Recall@1: did the top result match the ground truth?
- Embedding throughput: chunks per second during indexing
- RAM usage during embedding generation
- Model load time on first query
Results
| Model | Recall@10 | Recall@1 | Embed throughput | RAM during embed |
|---|---|---|---|---|
| mxbai-embed-large | 91% | 74% | 180 chunks/s | 1.8 GB |
| bge-m3 | 89% | 71% | 95 chunks/s | 2.9 GB |
| nomic-embed-text | 87% | 68% | 420 chunks/s | 0.7 GB |
| snowflake-arctic-embed-m | 85% | 65% | 390 chunks/s | 0.6 GB |
| all-minilm-l6-v2 | 76% | 51% | 1,100 chunks/s | 0.2 GB |
The performance gap between mxbai-embed-large and all-minilm-l6-v2 is large: 15 percentage points on Recall@10, 23 percentage points on Recall@1. For a RAG system where the quality of the retrieved context determines the quality of the generated answer, this is a meaningful difference.
all-minilm is in almost every "getting started with RAG" tutorial because it is small and fast. For prototyping, the speed matters. For production, the retrieval quality is what your users will notice.
The nomic-embed Tradeoff
nomic-embed-text stands out as the best performance-per-resource model in the group. At 87% Recall@10 with a 420 chunks/s throughput and a minimal RAM footprint in our runs, it is the practical default for self-hosted setups where memory is limited. The quality gap between nomic-embed and mxbai-embed-large (4 percentage points on Recall@10) is real but relatively small compared to the difference in resource requirements.
For most solo self-hosted RAG deployments, nomic-embed-text is the right starting point. If your evaluation shows the retrieval quality is insufficient for your use case, mxbai-embed-large is the next step up, and bge-m3 is worth testing if your documents are multilingual.
The Asymmetric Search Problem
One thing the headline numbers do not capture well: embedding models behave differently on asymmetric search (short queries against long documents) versus symmetric search (similar-length documents).
Code documentation has short queries ("how do I configure the retry policy?") against long chunks (full function documentation with examples). Models trained with this asymmetry in mind, like snowflake-arctic-embed, are specifically designed for this. In our test, snowflake-arctic-embed underperformed its expected ranking on this metric, but that result is specific to our corpus. On corpora with longer queries and shorter documents, the ranking may differ.
Run your own evaluation on a sample of your actual queries and documents. The rankings from public benchmarks reflect the training data of the benchmark, not necessarily your use case.
Chunking Strategy Interacts With the Model
The embedding model is not the only variable. How you split documents into chunks significantly affects retrieval quality, and the right chunking strategy depends partly on the model.
Models with larger context windows (nomic-embed-text supports up to 8,192 tokens per the Nomic documentation) can embed larger chunks, which can preserve more context for long documents. Models with smaller windows require tighter chunking. A chunk that gets truncated produces a different embedding than one that fits.
We ran a secondary test comparing smaller and larger chunk sizes using nomic-embed-text. Recall@10 improved on short-answer queries with the smaller chunks, and on synthesis queries the larger size performed better because the relevant information spanned multiple paragraphs. Neither was universally better.
The practical approach: start with a moderate chunk size and a small overlap, evaluate on your actual queries, and adjust based on where recall drops.
Switching Embedding Models in Production
One operational point worth planning for early: switching embedding models in production requires reindexing your entire corpus, because the new model produces vectors in a different space that is incompatible with the old index.
If you start with nomic-embed-text and later decide to upgrade to mxbai-embed-large, you need to re-embed all of your documents and rebuild the index. For a large corpus, this is a significant compute job.
A few practices that help:
Keep your raw documents and chunked text separate from the index. If you store only the vectors and lose the source chunks, you cannot reindex without re-processing the original documents.
Version your index alongside your model choice. When you upgrade the embedding model, treat it as a new index deployment and run both in parallel for a transition period rather than cutting over immediately.
Store the model name and version as metadata on each vector or index collection. When debugging retrieval quality issues months later, knowing which model produced the index is essential.
Conclusion
For self-hosted RAG in 2026, the practical recommendation is:
Start with nomic-embed-text. It runs comfortably on limited hardware, processes documents quickly, and provides good retrieval quality. It is the right choice until your evaluation shows a specific gap.
Upgrade to mxbai-embed-large if your Recall@1 numbers are inadequate and you have the RAM headroom. The quality improvement is real and justifies the resource cost for production systems where retrieval quality directly affects user-facing output.
Avoid all-minilm for production retrieval. It is appropriate for prototypes and learning, but the Recall@1 gap compared to other models in the group is large enough to be visible in output quality.
The most important investment you can make before choosing a model is building a retrieval evaluation harness: a set of queries with known-good answers that you can run against your index to measure actual recall. Without it, you are optimizing for benchmark numbers that may not reflect your documents or your users.
Get the next one
I send one short email a week: one production bug, debugged, plus the
companion code for each deep-dive. No spam, unsubscribe anytime.
Reader challenge: what embedding model are you using in production, and what Recall@10 are you seeing on your own evaluation set? Reply with the numbers.
Sources
- Nomic AI, nomic-embed-text-v1.5: https://www.nomic.ai/blog/posts/nomic-embed-text-v1
- Mixedbread AI, mxbai-embed-large-v1: https://www.mixedbread.ai/blog/mxbai-embed-large-v1
- BAAI, BGE-M3 paper: https://arxiv.org/abs/2402.03216
About the Author
Toc Am
Founder of AmtocSoft. Writing practical deep-dives on AI engineering, cloud architecture, and developer tooling. Previously built backend systems at scale. Reviews every post published under this byline.
Published: 2026-07-27 · Written with AI assistance, reviewed by Toc Am.
Get These In Your Inbox
Weekly deep-dives on AI engineering, no fluff. Join the newsletter →
Or grab the book ($39, ~100 pages) · Buy me a coffee
☕ Buy Me a Coffee · 🔔 YouTube · 💼 LinkedIn · 🐦 X/Twitter
No comments:
Post a Comment