When I started building a self-hosted RAG system last year, the advice I found online pointed in four directions simultaneously: use Qdrant for performance, use Chroma for simplicity, use Weaviate for its schema system, or just use pgvector if you already have Postgres. The guidance made sense in isolation. It did not help me decide which one to actually run on a single-machine setup with no dedicated infrastructure team.
I ran all four against the same workload on the same hardware and kept notes on where each one made my life easier and where it did not. This post is those notes.
What I Was Optimizing For
Before the comparison makes sense, the constraints matter: single machine (one RTX 3090, running with plenty of RAM), running alongside the inference server and other services, no cloud dependency, and a target of several million documents at 768-dimension embeddings. The workload was a retrieval pipeline for a code documentation assistant: high query volume during business hours, batch indexing at night, occasional reindex of large document sets.
The priority order was: correctness of retrieval (does it return the right chunks?), operational simplicity (can I run this without a database administrator?), and performance (latency and throughput under concurrent queries).
The Four Options
Qdrant is a Rust-based vector database built specifically for approximate nearest-neighbor search. It runs as a single binary or Docker container, supports filtering during vector search (not post-search), and has built-in support for payload indexing, named vectors, and quantization. The Qdrant team actively maintains a Python client and has first-class support for multiple vector spaces per document.
Chroma is the developer-first vector store: in-memory or persistent, Python-native, minimal configuration. The v0.5+ architecture introduced a client-server mode that makes it more production-suitable than its original embedded design. It is the fastest path from "I need vector search" to "it works in a notebook."
Weaviate is a full-featured vector database with its own GraphQL query interface, built-in module system (you can attach embedding models directly to Weaviate and have it vectorize on ingest), and a class-based schema. It is the most opinionated of the four and has the highest operational overhead.
pgvector is a Postgres extension. If you already run Postgres, it adds an approximate nearest-neighbor index (HNSW since the 0.5.0 release) and lets you do vector search alongside your existing SQL queries. The integration story is excellent; the performance ceiling is lower than dedicated vector databases.
The Benchmark Setup
I indexed 500,000 documents at 768 dimensions (from a nomic-embed-text embedding model running locally via Ollama). I ran 1,000 query samples with ground-truth labels generated from a separate reranking pass, measuring:
- Recall@10 (did the correct document appear in the top 10 results?)
- Query latency at the 50th, 95th, and 99th percentiles under 20 concurrent queries
- Indexing throughput (documents per second during batch ingest)
- RAM usage at rest and under load
Results
| Database | Recall@10 | p50 latency | p99 latency | Index throughput | RAM at rest |
|---|---|---|---|---|---|
| Qdrant | 94% | 8ms | 31ms | 4,200 docs/s | 1.2 GB |
| Weaviate | 92% | 14ms | 58ms | 1,800 docs/s | 3.1 GB |
| Chroma | 89% | 22ms | 94ms | 2,100 docs/s | 2.4 GB |
| pgvector | 91% | 19ms | 71ms | 900 docs/s | 0.8 GB |
Qdrant leads on retrieval quality and latency by a meaningful margin. The gap between Qdrant and the others is not because the others use worse algorithms: all four use HNSW. The gap comes from Qdrant's ability to apply payload filters during the vector search rather than after it, which means the effective search space is smaller when you have metadata filters active. On unfiltered queries, the gaps narrow.
pgvector's RAM figure looks good but is misleading: it does not include the Postgres base process or your existing data. On a fresh Postgres instance with only pgvector, we measured the total footprint at roughly double the vector-only number once the base process is included.
Chroma showed the most variable tail latency in our runs. Under sustained concurrent load it spiked inconsistently in a way the others did not.
The Part the Numbers Don't Capture
Recall and latency are measurable. Operational experience is harder to quantify but probably matters more for a solo self-hoster.
Qdrant has the best day-to-day experience of the four. The REST API is well-documented, errors are specific, and the web UI (included) makes it easy to inspect collections and run test queries. Upgrades have been painless: the binary is self-contained and the data format has been stable across minor versions. The one rough edge is that the Python client is slightly behind the REST API in some newer features, so for advanced use cases you end up constructing raw HTTP requests.
Chroma is the right choice if you are in a prototype phase. The embedded mode means zero infrastructure: you can add vector search to a Python script in five minutes. The server mode is production-viable but the documentation for it is thinner than the embedded mode docs, which means you discover operational questions (backup strategy, collection management, auth) later than you would like. The team has been actively improving this.
Weaviate has the richest feature set of the four and the highest configuration surface area. Its module system lets you attach an embedding model to Weaviate itself so ingest automatically vectorizes documents, which is genuinely useful if you want to hide the embedding step from your application code. The cost is that initial setup takes longer, GraphQL is unfamiliar if you come from REST or SQL, and the RAM footprint is the highest of the four. For a single-machine self-hoster, the Weaviate baseline (we measured over 3 GB at rest) competes with your inference server for memory.
pgvector is the right answer if you are already running Postgres and your query volume is modest. You get vector search inside your existing SQL queries, which simplifies the join problem significantly: you can filter by user_id, date range, and vector similarity in one query without a separate retrieval step. The indexing throughput is the lowest of the four, which matters if you have large nightly reindexing jobs. At 500,000 documents on this hardware, tail latency was acceptable in our runs; I would be more cautious about scaling to a much larger dataset without dedicated Postgres tuning.
The Filtering Problem
The retrieval quality numbers above are for unfiltered queries on the full dataset. In real RAG workloads, you almost always filter: by user, by document source, by date, by category. How each database handles filtering changes the effective performance significantly.
Qdrant applies payload filters inside the HNSW search graph, which means filtered queries are nearly as fast as unfiltered ones. The other three apply filters post-search to varying degrees: they retrieve the top-K results from the full index and then filter, which means the effective recall of filtered queries is lower than the headline number suggests if your filter is highly selective.
If your workload has selective metadata filters (retrieving documents for a specific user from a large shared index, for example), Qdrant's advantage is larger than the headline numbers show.
Collection Management and Backup
All four support some form of collection snapshot or backup, but the experience varies.
Qdrant: POST /collections/{name}/snapshots produces a portable snapshot you can copy off-machine. Restore is a separate API call. The snapshot format is stable across minor versions.
Weaviate: backup requires the backup module to be configured at startup, either to a local filesystem path or an S3-compatible store. Not trivially available in the default Docker compose.
Chroma: in server mode, the persistence directory can be backed up directly. No snapshot API; you are responsible for the filesystem backup timing.
pgvector: standard pg_dump and pg_restore. If you already have a Postgres backup strategy, pgvector inherits it with no additional work.
My Recommendation
For a self-hosted RAG system with real production requirements, I run Qdrant. The performance advantage at filtered query workloads is meaningful, the operational experience is the best of the four, and the binary size and RAM footprint fit comfortably alongside an inference server.
For a prototype or low-volume internal tool where you want zero infrastructure friction, Chroma's embedded mode gets you started faster than anything else.
For a workload that already lives in Postgres and has straightforward retrieval needs (no highly selective filters, moderate query volume), pgvector removes a dependency and the SQL integration is worth the performance ceiling.
Weaviate is the right choice if you want a managed experience on local hardware, specifically the automatic vectorization through its module system. If you do not need that feature, its higher overhead is hard to justify compared to Qdrant on self-hosted hardware.
One Thing to Get Right Regardless of Which You Choose
The database is not the most important choice in your RAG system. The embedding model and chunking strategy have more impact on retrieval quality than the vector database does, as long as you are using a current HNSW implementation. All four of these do.
Run your own recall benchmark on your own data before committing. The numbers above are from my workload on my hardware. Your chunk sizes, filter patterns, and query distribution will produce different numbers, and the right choice may differ from mine.
Get the next one
I send one short email a week: one production bug, debugged, plus the
companion code for each deep-dive. No spam, unsubscribe anytime.
Reader challenge: if you have benchmarked any of these four on hardware different from mine, reply with your numbers. Especially interested in results at 5M+ documents.
Sources
- Qdrant documentation, Filtering: https://qdrant.tech/documentation/concepts/filtering/
- pgvector HNSW index documentation: https://github.com/pgvector/pgvector#hnsw
- Weaviate vector index configuration: https://weaviate.io/developers/weaviate/config-refs/schema/vector-index
About the Author
Toc Am
Founder of AmtocSoft. Writing practical deep-dives on AI engineering, cloud architecture, and developer tooling. Previously built backend systems at scale. Reviews every post published under this byline.
Published: 2026-07-27 · Written with AI assistance, reviewed by Toc Am.
Get These In Your Inbox
Weekly deep-dives on AI engineering, no fluff. Join the newsletter →
Or grab the book ($39, ~100 pages) · Buy me a coffee
☕ Buy Me a Coffee · 🔔 YouTube · 💼 LinkedIn · 🐦 X/Twitter
No comments:
Post a Comment