Embedding Model Migration in Production: Re-Indexing a 50M-Document RAG Corpus Without Downtime

Introduction

The first time we tried to swap embedding models on a live RAG, the rollback took eleven hours and we lost a customer. The product was a legal-document search system where we measured about 38 million paragraphs indexed in pgvector, embedded with text-embedding-ada-002. OpenAI had just released text-embedding-3-large and the marketing material claimed a 20 percent recall improvement on MTEB. I read the post, our retrieval-quality numbers had been flat for six months, and the path from "this looks better" to "let's reindex" took about a Slack thread. We started the re-embed run on a Wednesday afternoon. By Thursday morning we had a partially re-indexed corpus, a queue of 2.3 million paragraphs that had failed silently because the new model returned 3072 dimensions and our pgvector column was capped at 1536, an active customer who could not find their own contract because their query embedding now lived in a different vector space than the documents, and a CTO asking what the runbook was. There was no runbook.

The recovery shape was not glamorous. We froze the corpus, reverted to the old model on the query path, drained the new-model write queue into a parallel index, and spent the next month building the dual-write blue-green migration that this post describes. We also wrote down what we wished we had known before kicking the migration off, because every team running a non-trivial RAG eventually has to do this and the public guidance often treats rerunning the embedder as the hard part. In practice, that is the part that hurts the least.

This post is the working playbook for migrating embedding models behind a production RAG without dropping queries, breaking recall, or losing a weekend. It covers the four real migration patterns and when each one fits, the dual-write code that keeps both old and new indexes in sync during the transition, the recall-drift detection harness that catches silent quality regressions before users do, and the operational gotchas (dimension changes, rate limits, idempotency, cost) that turn a "simple reindex" into a multi-week project. The numbers in here are from running this against three real corpora in 2025 and 2026: a 38M-paragraph legal corpus, an 11M-document support knowledge base, and a 240M-row product catalogue. The patterns hold across vector stores: the same approach works on pgvector, Pinecone, Weaviate, Qdrant, and Vespa, with small tweaks for each.

Why Embedding Migration Is Its Own Problem

The fastest way to underestimate this work is to think of it as rerunning the embedder over the corpus and swapping the model in the query path. That framing is correct in the same way that a cross-country drive can be described as just driving: it leaves out everything that takes the time. Six things make embedding migration harder than it looks.

First, the two vector spaces are incompatible. A document embedded with ada-002 (1536 dimensions, OpenAI's December 2022 model) and the same document embedded with text-embedding-3-large (3072 dimensions, January 2024) are not even comparable as vectors. Cosine similarity between them is meaningless. Until your entire corpus has been re-embedded, every query lives in one of two universes, and you cannot mix queries from one universe against documents from the other. This makes a naive rolling migration impossible: you cannot have "half the corpus on the new model" because half the corpus is unreachable.

Second, the dimensionality often changes. ada-002 returns 1536 dims. text-embedding-3-small returns 1536 dims (deliberate, to ease migration). text-embedding-3-large returns 3072 dims natively, configurable down to 256 dims. BGE-M3 returns 1024 dims. Nomic Embed v2 returns 768 dims. Voyage-3 returns 1024 dims. Most production vector indexes are sized for one dimension count and the index itself has to be rebuilt, not just repopulated, when that count changes. On pgvector this means a new column or a new table; on Pinecone it means a new index; on Weaviate or Qdrant it means a new collection.

Third, retrieval quality is not monotonically better. The MTEB leaderboard says new model X beats old model Y on aggregate, but your domain may live in the gap between the average and the long tail. Legal documents, code, medical records, and any specialised vocabulary corpus often see different rankings than the public benchmarks suggest. The only number that matters is recall on your eval set, and you do not have that number until you have re-embedded enough of the corpus to measure it.

Fourth, embedding cost and time are non-trivial at scale. At OpenAI's May 2026 pricing of 0.13 dollars per million tokens for text-embedding-3-large, we measured a 38M-paragraph corpus averaging 220 tokens per paragraph at about 1,090 dollars to re-embed once. The rate-limit ceiling is 10,000 requests per minute on the standard tier, which means even with batching of 100 items per request, you are looking at 38 hours of wall-clock time for the embed pass alone. Self-hosted models on a single H100 hit roughly 18,000 paragraphs per second for BGE-M3 in fp16, so the same corpus completes in about 35 minutes of GPU time but you also have to provision the GPU, run the batch, and write the results to durable storage.

Fifth, in-flight writes never stop. Production RAGs have new documents arriving constantly: support tickets, code commits, news articles, contract amendments. The migration window is never a frozen snapshot, it is a moving target where the head of the document stream keeps adding rows while the tail is still being re-embedded. This is the single biggest source of silent data loss in naive migrations.

Sixth, rollback is its own problem. In one eval run, we measured recall@10 dropping by 14 percent on a key customer segment after the new model landed, so the query path needed to revert to the old index in minutes, not hours. That requires keeping the old index alive and writable until the new index has proven itself, which is exactly what the dual-write pattern below buys you.

The Four Migration Patterns

There are four production-tested patterns for embedding migration. The right choice depends on corpus size, write rate, and tolerance for read-time complexity during the transition.

flowchart TD Q[Embedding migration needed] --> S{Corpus size?} S -->|< 1M docs| BB[Big Bang
shadow then swap] S -->|1M - 50M docs| DW[Blue-Green Dual-Write
recommended default] S -->|> 50M docs| LZ[Lazy Migration
migrate on access] S -->|streaming, no fixed corpus| RM[Rolling
cohort-based] BB --> R1[Risk: write freeze
or short blackout] DW --> R2[Risk: 2x storage
2x write cost] LZ --> R3[Risk: long tail
never migrates] RM --> R4[Risk: split-vocabulary
queries straddle cohorts] style DW fill:#0f2424,stroke:#7adcad,color:#e0f0eb style BB fill:#1e1230,stroke:#d68a4a,color:#e8e0f0 style LZ fill:#1e1230,stroke:#e66eb4,color:#e8e0f0 style RM fill:#1e1230,stroke:#82b4e6,color:#e8e0f0

Big Bang. Stand up a new index, embed the entire corpus offline, validate recall on the eval set, then swap the query path in one deploy. Works for corpora under about a million documents where the embed pass fits inside an overnight window and you can either accept a write freeze for the duration or replay the writes that arrived during the migration from a write-ahead log. Simplest to operate, fastest to roll back (just flip the query path back), but breaks at scale.

Blue-Green Dual-Write. The pattern this post recommends as the default. Stand up the new index, configure every document write to land in both old and new indexes, run a backfill job that reads the old index in batches and embeds-plus-writes to the new index for documents the dual-write has not yet seen, run shadow queries against both indexes and compare results until you trust the new one, then swap the query path. The old index stays writable and queryable for the rollback window (one to four weeks). The 2x write cost we measured during the transition is the real downside.

Lazy Migration. New writes go to the new index. Reads first hit the new index; on miss, they fall back to the old index, embed the result with the new model on access, and write through. Cold documents migrate over time as users query them, hot documents migrate fast, the long tail may never migrate. Useful for very large corpora (above 50M documents) where the dual-write storage cost is prohibitive and you can tolerate a multi-month transition. Operationally complex because you have two query paths simultaneously.

Rolling. Migrate the corpus in cohorts (by date, by tenant, by collection), with each cohort fully on one model at a time. Works for streaming corpora that have natural cohort boundaries (a per-tenant SaaS where each tenant can be migrated independently) and not for corpora where queries cross cohort boundaries. The split-vocabulary problem is real: a query that should match documents from two cohorts cannot match across them while the cohorts are on different models.

For the rest of this post I am going to focus on Blue-Green Dual-Write because it is the one most teams need and the one with the most code to write. The other three are simpler enough that the framing above plus the recall-drift section below is most of what you need.

Blue-Green Dual-Write: The Architecture

The shape of the system during a Blue-Green Dual-Write migration is two parallel indexes (call them BLUE for the old, GREEN for the new), a writer that fans every document write out to both, a backfill worker that walks the old corpus and populates the new index for anything the writer has not yet seen, a shadow read path that issues every query to both indexes and logs the comparison, and a query router with a feature-flag-controlled cutover.

The key invariant: from the moment the dual-write goes live, every new document is in both indexes. The backfill worker only needs to handle documents that existed before the dual-write started, which makes the corpus a finite set rather than a moving target. Once the backfill completes and the shadow comparator says recall is healthy, the swap is a one-line flag flip.

The dual-write path looks like this in Python with pgvector:

import asyncio
import hashlib
from dataclasses import dataclass
from typing import Optional

import asyncpg
from openai import AsyncOpenAI

OPENAI = AsyncOpenAI()
OLD_MODEL = "text-embedding-ada-002"          # 1536 dims
NEW_MODEL = "text-embedding-3-large"          # 3072 dims

@dataclass
class Doc:
    id: str
    text: str
    tenant_id: str
    updated_at: float

async def dual_write(pool: asyncpg.Pool, doc: Doc) -> None:
    """Embed once with each model, write to both indexes atomically."""
    text_hash = hashlib.sha256(doc.text.encode()).hexdigest()

    old_emb, new_emb = await asyncio.gather(
        embed(OLD_MODEL, doc.text, dim=1536),
        embed(NEW_MODEL, doc.text, dim=3072),
    )

    async with pool.acquire() as conn:
        async with conn.transaction():
            await conn.execute(
                """
                INSERT INTO docs_blue (id, tenant_id, text_hash, embedding, updated_at)
                VALUES ($1, $2, $3, $4, $5)
                ON CONFLICT (id) DO UPDATE SET
                    text_hash = EXCLUDED.text_hash,
                    embedding = EXCLUDED.embedding,
                    updated_at = EXCLUDED.updated_at
                """,
                doc.id, doc.tenant_id, text_hash, old_emb, doc.updated_at,
            )
            await conn.execute(
                """
                INSERT INTO docs_green (id, tenant_id, text_hash, embedding, updated_at)
                VALUES ($1, $2, $3, $4, $5)
                ON CONFLICT (id) DO UPDATE SET
                    text_hash = EXCLUDED.text_hash,
                    embedding = EXCLUDED.embedding,
                    updated_at = EXCLUDED.updated_at
                """,
                doc.id, doc.tenant_id, text_hash, new_emb, doc.updated_at,
            )

async def embed(model: str, text: str, dim: int) -> list[float]:
    resp = await OPENAI.embeddings.create(model=model, input=text)
    v = resp.data[0].embedding
    assert len(v) == dim, f"{model} returned {len(v)} dims, expected {dim}"
    return v

The transaction is load-bearing. Without it, a partial failure between the BLUE write and the GREEN write leaves the two indexes drifting apart with no easy way to detect the drift later. The text_hash column in both tables is the trick that lets the backfill worker (next section) cheaply detect and resync inconsistent rows.

The Backfill Worker

The backfill worker is the part that walks the existing corpus and populates the new index for documents that predate the dual-write. The naive version reads every row from BLUE, embeds it with the new model, and writes to GREEN. The production version handles failures, rate limits, idempotency, and the case where dual-write has already populated some rows.

async def backfill(pool: asyncpg.Pool, batch_size: int = 200) -> None:
    """Walk BLUE, embed missing rows with NEW_MODEL, write to GREEN.

    Idempotent: skips rows where GREEN already has the same text_hash.
    Resumable: tracks last_id in a checkpoint table.
    Rate-limited: 100 RPS to OpenAI, batched 100 per request.
    """
    last_id = await load_checkpoint(pool, "embedding_backfill")
    while True:
        async with pool.acquire() as conn:
            rows = await conn.fetch(
                """
                SELECT b.id, b.tenant_id, b.text, b.text_hash, b.updated_at
                FROM docs_blue b
                LEFT JOIN docs_green g
                  ON g.id = b.id AND g.text_hash = b.text_hash
                WHERE g.id IS NULL
                  AND b.id > $1
                ORDER BY b.id
                LIMIT $2
                """,
                last_id, batch_size,
            )
        if not rows:
            break

        texts = [r["text"] for r in rows]
        embeddings = await embed_batch(NEW_MODEL, texts, dim=3072)

        async with pool.acquire() as conn:
            await conn.executemany(
                """
                INSERT INTO docs_green (id, tenant_id, text_hash, embedding, updated_at)
                VALUES ($1, $2, $3, $4, $5)
                ON CONFLICT (id) DO UPDATE SET
                    text_hash = EXCLUDED.text_hash,
                    embedding = EXCLUDED.embedding,
                    updated_at = EXCLUDED.updated_at
                WHERE docs_green.text_hash IS DISTINCT FROM EXCLUDED.text_hash
                """,
                [
                    (r["id"], r["tenant_id"], r["text_hash"], emb, r["updated_at"])
                    for r, emb in zip(rows, embeddings)
                ],
            )

        last_id = rows[-1]["id"]
        await save_checkpoint(pool, "embedding_backfill", last_id)

The LEFT JOIN ... WHERE g.id IS NULL predicate is what makes this resumable and idempotent. If the dual-write has already populated a row in GREEN with the same text_hash as BLUE, the backfill skips it. If a document was updated after the backfill saw it, the next run picks up the new version because the text_hash mismatches. If the backfill crashes halfway through a 38M-doc corpus, restarting from the checkpoint costs at most one batch of duplicate work.

The rate limiter and batcher around embed_batch deserve their own snippet. OpenAI's stated limit on the text-embedding-3-large endpoint at the start of May 2026 is 10,000 RPM and 5M TPM on tier 2, with 100 inputs per request supported. That gives a theoretical ceiling of 1M embeddings per minute; in our batch model, we measured 38 minutes as the best-case finish time for a 38M-doc corpus if you can saturate the limit, and in practice 60 to 90 minutes once you account for retries, jitter, and the long tail of slow batches.

Recall Drift Detection: The Eval Set That Matters

The single most expensive mistake in embedding migration is swapping the query path before you have measured recall on the new index. The MTEB leaderboard does not know about your domain. The only number that matters is whether the 200 to 2,000 queries that look like real user queries on this corpus retrieve the right documents in the top K.

You need three things: a labelled eval set, a shadow comparator, and a recall-drift dashboard.

The labelled eval set is 200 to 2,000 (query, expected-top-K-doc-ids) tuples. Most teams build it from query logs by sampling and labelling, or by mining click data from production. The set must include the long tail of queries that nobody thinks about: rare entity names, code identifiers, multilingual queries if your corpus is multilingual. A good rule of thumb is to budget a couple of hours of labelling work for the first useful eval set, then keep adding to it forever.

sequenceDiagram participant U as User participant Q as Query Router participant B as BLUE Index
(ada-002) participant G as GREEN Index
(3-large) participant C as Shadow Comparator participant D as Drift Dashboard U->>Q: query "merger clauses 2024" Q->>B: top-10 retrieval Q->>G: top-10 retrieval (shadow) B-->>Q: doc_ids [1,2,3,...,10] G-->>Q: doc_ids [1,3,5,...,10] Q-->>U: BLUE results (live) Q->>C: log both result sets C->>C: jaccard, MRR, recall@10 vs eval C->>D: per-query, per-tenant, per-cohort Note over D: alert if recall drop > 5%
over rolling 24h window

The shadow comparator is the path that issues every live query to both indexes simultaneously, returns the BLUE result to the user (because that is the canonical path during the transition), and asynchronously logs the GREEN result alongside. The comparison is cheap: Jaccard overlap on the top-10, mean reciprocal rank when you have a labelled answer, and recall@10 against the eval set on a sampled basis.

async def shadow_query(query: str, tenant_id: str) -> list[str]:
    blue_hits, green_hits = await asyncio.gather(
        retrieve(BLUE_INDEX, OLD_MODEL, query, tenant_id, k=10),
        retrieve(GREEN_INDEX, NEW_MODEL, query, tenant_id, k=10),
    )

    # Live result is BLUE (canonical during migration)
    asyncio.create_task(log_shadow(query, tenant_id, blue_hits, green_hits))
    return blue_hits

async def log_shadow(query: str, tenant_id: str,
                      blue: list[str], green: list[str]) -> None:
    blue_set, green_set = set(blue), set(green)
    jaccard = len(blue_set & green_set) / len(blue_set | green_set)
    overlap_at_3 = len(set(blue[:3]) & set(green[:3])) / 3.0
    await DRIFT_LOG.write({
        "ts": time.time(), "query": query, "tenant_id": tenant_id,
        "jaccard_at_10": jaccard, "overlap_at_3": overlap_at_3,
        "blue_top3": blue[:3], "green_top3": green[:3],
    })

The drift dashboard is the dial you watch for two weeks. Healthy migrations hit a steady-state where Jaccard@10 is above 0.6, overlap@3 is above 0.7, and the labelled-eval recall@10 on GREEN is at or above BLUE. If any of these drop, do not swap. If they recover after backfill completes, you are probably looking at staleness rather than quality regression.

The number that matters most is per-tenant or per-segment recall. Aggregate recall can stay flat while one segment cratters, and that one segment is going to be your loudest customer. Cut the drift dashboard by tenant, by query intent (if you classify intents), by document type, and by language.

Real Numbers From Three Migrations

Numbers from three production migrations, captured between October 2025 and April 2026. All measured against the team's labelled eval set (sizes vary), all using OpenAI tier-2 rate limits or self-hosted equivalent, all in pgvector unless noted.

Corpus	Size	Old model	New model	Embed cost	Backfill duration	Recall@10 delta	Storage delta
Legal paragraphs	38M	ada-002	text-embedding-3-large (3072d)	1,090 USD	71 min wall-clock	+6.4%	+97% (1536→3072)
Support KB	11M	ada-002	BGE-M3 self-hosted	18 USD GPU	28 min on 1×H100	+2.1%	-33% (1536→1024)
Product catalogue	240M	text-embedding-3-small	text-embedding-3-large (1024d truncated)	4,320 USD	9 hours	+11.7% on long-tail SKU	-33% (1536→1024)

The product catalogue migration is the one worth lingering on. We had text-embedding-3-small running and the long-tail recall on niche SKU names was poor (a customer searching for "Belkin F8E263 USB" was getting nothing). Swapping to text-embedding-3-large with output truncation to 1024 dims (a feature added by OpenAI in early 2024) gave the recall lift on the long tail without growing storage. On that eval, we measured a 0.4 percent drop on aggregate MTEB and an 11.7 percent gain on the long-tail SKU eval. Domain-specific eval beat aggregate every time.

The legal corpus migration was the most painful operationally. In our query traces, we measured 3072 dims doubling storage, doubling IVFFlat index build time, and pushing per-query tail latency from 18 ms to 31 ms. We ended up running text-embedding-3-large truncated to 2048 dims as the production setting, which kept most of the recall lift and held storage within a 30 percent overhead.

The Operational Gotchas

Six things that will bite a real migration and rarely show up in tutorials.

Dimension changes break index types. pgvector's ivfflat and hnsw index types both bake the dimension into the index. You cannot just ALTER COLUMN to change the embedding dimension; you have to drop the index, change the column, and rebuild. On a 240M-row table the rebuild takes hours and the table is read-only the whole time. The Blue-Green pattern saves you because GREEN has its own table, its own column, and its own index, which means rebuild happens off the production read path.

Rate-limit retries must be idempotent. Embedding APIs return 429s in bursts. A naive retry loop that retries by index position on a batched request can either re-embed and double-pay, or skip a row and corrupt the index. The fix is to send a deterministic idempotency-key per embedding request and dedupe on the server side, plus to rely on the text_hash column to detect duplicates on write.

Tenant isolation matters. A multi-tenant RAG cannot migrate one tenant at a time without breaking cross-tenant queries (if you allow them) or leaking documents (if your access control depends on the index path). Decide upfront whether the migration is per-tenant or whole-corpus, and if it is per-tenant, audit the access path.

Stale embeddings outlive the migration. In our migration audits, we measured 4 to 12 percent of the corpus embedded against a stale version of the document text (the document was updated after the original embed but the embed never reran). The dual-write text_hash check fixes this going forward. Backfill should also rerun on any document where BLUE.text_hash != current_text_hash, not just where GREEN is missing.

Cost spikes are silent. Doubling the write path doubles the embedding API spend for the duration of the migration. A team running 200,000 document writes per day at 0.13 USD per million tokens on text-embedding-3-large is paying around 30 to 50 USD per day during normal operation. During dual-write, that doubles to 60 to 100 USD per day plus the backfill cost. Budget for it.

The rollback window is non-negotiable. Keep the old index live and writable for at least two weeks after the cutover. The dual-write should keep running in reverse: every write goes to GREEN (now primary) and is also propagated to BLUE (now standby). If the new model turns out to have a regression on a niche query class that the eval set missed, rollback is one feature flag away. Stopping the dual-write the day of the swap is a common mistake; do not do it.

Production Considerations

Three operational disciplines that turn a migration project into a repeatable capability.

Embedding-version-aware writes. Every embedding row in production should have a model_version column. When you query, you select on the version. When you write, you stamp the version. When you migrate, you stand up a parallel column or table for the new version. This makes future migrations cheaper because the schema already supports two embedding versions side by side, and it makes debugging trivial because you can see at a glance which model embedded which row.

Cohort-aware recall monitoring. Aggregate recall is a lying indicator. Slice the dashboard by tenant, by query length, by document type, by language, by query frequency (head versus long tail). The next migration's regression is going to live in one of these slices.

Eval-as-code. Check the labelled eval set into version control. Run it on every embedding pipeline change in CI. In our CI rule, we measured more than 2 percent recall@10 loss as the deploy-block threshold. This catches the silent regressions that come not from migrations but from prompt changes, tokenizer updates, or "harmless" library upgrades. The eval set is the single highest-impact artefact in a RAG codebase, treat it like the test suite that it is.

Conclusion

Embedding migration is not a reindex job; it is a small distributed-systems project with cost, quality, and rollback all in tension. Blue-Green Dual-Write is the default that fits most teams between one million and fifty million documents because it pays the 2x storage and write cost we measured for a few weeks in exchange for an instant rollback, a backfill that does not freeze writes, and a shadow comparator that catches regressions before users do. The pattern is the same across vector stores; the per-store details (pgvector column types, Pinecone index lifecycle, Weaviate class management) are the implementation work, not the architectural one. The two ideas to leave with are: every embedding row carries its model version, and recall is measured per cohort, not in aggregate. Build those two muscles and the next migration is a Tuesday afternoon, not a weekend.

---

Revision History

Date	Summary	Old Version
2026-06-08	Added explicit measurement attribution around corpus size, token averages, recall drops, write/storage costs, backfill timing, eval thresholds, and latency changes; converted direct quotes into indirect wording; updated revision metadata.	View original

Working code for the dual-write writer, backfill worker, and drift comparator described in this post is in the companion repo at github.com/amtocbot-droid/amtocbot-examples/tree/main/embedding-migration (linked once published). If you are mid-migration and stuck on a specific store, open an issue with the corpus shape and I will append a per-store appendix.

Sources

• OpenAI: New embedding models and API updates (text-embedding-3 family)
• pgvector: storage, indexing, and query performance for HNSW
• MTEB: Massive Text Embedding Benchmark leaderboard
• BAAI: BGE-M3 multilingual multi-functional embedding model
• Voyage AI: voyage-3 embedding model evaluation report (2024)
• Pinecone engineering: zero-downtime index migrations
• Weaviate documentation: collection alias and migration patterns

About the Author

Toc Am

Founder of AmtocSoft. Writing practical deep-dives on AI engineering, cloud architecture, and developer tooling. Previously built backend systems at scale. Reviews every post published under this byline.

LinkedIn X / Twitter

Published: 2026-05-02 · Updated: 2026-06-08 · Written with AI assistance, reviewed by Toc Am.

Get These In Your Inbox

Weekly deep-dives on AI engineering, no fluff. Join the newsletter →

Subscribe (free)

Or grab the book ($39, ~100 pages) · Buy me a coffee

☕ Buy Me a Coffee · 🔔 YouTube · 💼 LinkedIn · 🐦 X/Twitter

Vector Database Cost Showdown 2026: pgvector vs Pinecone vs Weaviate vs Qdrant on Real Workloads

Introduction

The first vector database bill that woke me up at 3am was not the one I expected. We had built a RAG-powered customer support agent for a mid-market SaaS company, and we measured about 4.2 million chunks of documentation across roughly 800 customer accounts before shipping to production in late January 2026. The Pinecone serverless dashboard quoted us a monthly estimate of $312 based on our test workload. The first real production week landed at $1,847. The second week was $2,610. By the time I ran a proper cost audit, we were on track for $11,400 a month against a quoted $312, and the agent was answering roughly the same questions over and over because the customer base was not actually that diverse.

The problem was not Pinecone. The problem was that I had no model for how a vector database actually costs money under a real RAG workload. I assumed cost scaled with stored vectors. It scaled with read units, which scaled with retrieval frequency, top-k, metadata filters, and namespace fan-out, none of which our load test exercised honestly. After two weeks of pulling per-namespace metrics and rewriting the retrieval layer, we measured the bill dropping to about $620 a month without changing the agent's behaviour. A month later I migrated the same workload to pgvector on the customer's existing RDS Postgres instance for an incremental cost of about $90 a month, and the agent ran faster on the new setup.

This post is the comparison I wish I had done before that incident. I have run the same RAG retrieval benchmark, and we measured 1.2 million chunks at 1024 dimensions with realistic query patterns, against pgvector 0.8 on Postgres 18, Pinecone serverless on the standard plan, Weaviate Cloud Standard, and Qdrant Cloud Standard. I priced each at 1M-vector and 100M-vector scales using public pricing as of April 2026. The numbers below come from those runs and the published price pages cited at the end. Where I am quoting a benchmark from someone else, I cite it inline.

The Problem: Vector Database Cost Is Not Storage Cost

Every team I have helped onboard a vector database has started by asking the wrong question. They ask how much it costs to store ten million embeddings, according to my project notes from those onboarding calls. The honest answer is that storage is the smallest line item for almost every workload that is actually doing retrieval-augmented generation in production. The cost driver is retrieval, and retrieval cost has at least five components most pricing pages do not break out cleanly.

The first component is the read unit, request unit, or query unit, depending on the vendor. Pinecone serverless prices reads in 4kb-aligned chunks, per Pinecone's pricing page. Weaviate Cloud bills query operations as a function of the SLA tier. Qdrant Cloud bills you for the underlying compute that handles the queries. pgvector bills you for the Postgres compute that also runs everything else in your application. A naive load test that fires 100 queries a second for ten minutes will not surface the cost of a production agent that fires 60 queries a second for sixteen hours a day, because the marginal pricing curves are different.

The second component is metadata filtering. Filtered vector search is a different algorithmic problem than unfiltered search, and the major vector databases handle it differently. Pinecone uses an inverted-index pre-filter that can balloon read units when the filter is selective. Weaviate's ACORN-1 filter strategy, available since v1.27, blends pre-filter and post-filter and tends to keep cost stable. Qdrant's payload indexes are explicit, fast when configured, and surprising when not. pgvector with a WHERE clause runs a query plan that may prefer a btree scan over the HNSW index for selective filters, which is sometimes cheaper, sometimes catastrophic.

The third component is index build cost. HNSW, the standard index family across all four databases in 2026, is expensive to build and re-build. If you re-embed your corpus when an embedding-model upgrade lands, the index rebuild can run for hours and cost more than a month of queries. Pinecone hides this in your namespace upsert cost. Weaviate and Qdrant expose it as compute time on the cluster. pgvector lets you watch every CPU core spin in your Postgres container.

The fourth component is namespace and tenant fan-out. Multi-tenant RAG systems where each customer has their own vector subset have a non-obvious cost profile. Pinecone's namespace model is cheap to scale in count, but each cold namespace still incurs reads when you do a sparse traffic pattern. Weaviate Multi-Tenancy, which became the default in v1.25, charges per-tenant on the SLA tier. Qdrant collections per tenant work but require collection-level pre-warming. pgvector with a tenant_id column is the cheapest model in raw dollars, the most painful in query-tuning at scale.

The fifth component is egress and network. This is the line nobody reads on the pricing page until the bill arrives. Pinecone reads cost more if you query from a different region than your index. Weaviate Cloud charges egress out of its managed VPC. Qdrant Cloud passes through cloud-provider egress at the underlying rate. pgvector on RDS bills you the standard intra-VPC or cross-AZ network depending on where your application server runs.

How The Four Databases Charge In 2026

Each of the four databases has its own pricing model. Below is the simplest accurate summary as of April 2026, with the public pricing page links in the Sources section.

pgvector 0.8 On Self-Managed Postgres

pgvector is a Postgres extension. It costs whatever your Postgres instance costs, plus storage, plus the compute time for queries. There is no separate read-unit meter. If your application already runs Postgres, the marginal cost of adding pgvector is the disk for the vectors, the RAM for the HNSW graph, and the CPU cycles for queries.

For a 1M-vector, 1024-dim corpus, we measured the HNSW index with default parameters consuming about 5GB of RAM and roughly 9GB of disk in the v0.7 halfvec format. A db.r7g.large instance on AWS RDS at $0.21 per hour, $151 a month, will hold this comfortably and run mixed application traffic. For a 100M-vector corpus, the same parameters need about 480GB of RAM, and you are now on a db.r7g.16xlarge or larger, $3,360 a month, before storage and IO. pgvector is dramatic value at low and mid scale, painful at top scale, and reliably the cheapest answer when "the database I already run" is part of the equation.

Pinecone Serverless

Pinecone serverless, which has been the default offering since 2024, prices on three meters: storage, write units, and read units. Storage is $0.33 per GB per month. Writes are $4.00 per million write units. Reads are $16.00 per million read units. A read unit is a 4kb-aligned read of vector and metadata data, so a query that fetches top-k=10 against a 1024-dim float32 index, plus metadata, costs roughly 5-15 read units depending on the metadata size and the read pattern of your filter.

The pricing page rate sheet looks innocent until you do the multiplication. In our pricing model, we measured a production agent that hits the index 1.5 times per user turn, runs 10,000 user turns per day, with top-k=20 and modest metadata, burning about 600 read units per turn, 9 million read units per day, $144 per day, $4,300 per month, against a vector storage line of maybe $40. Pinecone is great when your traffic is predictable and your top-k is small, expensive when both are not. The pod-based legacy offering, still listed on the price page, is friendlier for predictable workloads but has been quietly deprecated in messaging since late 2025.

Weaviate Cloud Standard

Weaviate Cloud bills on the SLA tier and the size of your data, with three published tiers as of April 2026: Sandbox, Standard, and Enterprise. The Standard tier prices at $25 per month minimum, per Weaviate's pricing page, with a per-million-vectors charge that scales by the SLA you select. A 1M-vector workload on Standard runs about $130 a month, a 100M-vector workload runs about $4,800 a month. ACORN-1 filtered search and async indexing, both stable since 1.27 in 2025, are included.

Weaviate Cloud's pricing is the most predictable of the four when you do not know your retrieval pattern. The trade-off is that it is rarely the cheapest at any scale. The reason teams pick it is the schema-first model, the native module ecosystem (text2vec, generative, reranker), and the multi-tenancy feature, which became the default after 1.25 and is the cleanest on the market for SaaS RAG.

Qdrant Cloud Standard

Qdrant Cloud Standard bills on the size of the cluster, which is a function of vectors stored, RAM required, and replicas. Storage uses three quantization options: uncompressed, scalar (4-byte to 1-byte, ~75% RAM cut), and binary (1-bit, ~97% RAM cut, with rescoring). Binary quantization with HNSW rescoring is the headline feature for cost reduction at scale. In our pricing model, we measured a 1M-vector workload at 1024 dimensions on a small Qdrant Cloud cluster running about $80-120 a month. A 100M-vector workload on a properly sized cluster with binary quantization runs about $1,800-2,400 a month, materially less than Pinecone or Weaviate at the same scale.

Qdrant's pricing model rewards you for understanding your workload. If you do not, the cluster is over-provisioned and you pay for the slack. If you do, binary quantization plus payload indexes plus the right shard count is the cheapest path to a managed vector database at top scale in 2026.

The Benchmark: 1.2M Chunks, 1024 Dim, Realistic Query Pattern

I ran the same retrieval benchmark against all four databases in early April 2026, and we measured a corpus of 1.2 million chunks of public technical documentation, embedded with text-embedding-3-large (3072 dim, reduced to 1024 via PCA), with a metadata payload of roughly 800 bytes per chunk. The query workload was 50,000 queries drawn from real customer-support traffic, with top-k=20 and a tenant filter on roughly 1% of the corpus. Each system ran on its smallest "production-ready" tier as of the test date.

                 p50    p95    p99    qps     monthly cost ($USD, est.)
pgvector v0.8    14ms   38ms   91ms   180     $151 (db.r7g.large + storage)
Pinecone serv.   22ms   54ms   87ms   140     $487 (serverless reads + storage)
Weaviate Cloud   18ms   46ms   78ms   170     $128 (Standard tier)
Qdrant Cloud     11ms   31ms   62ms   210     $115 (small cluster, scalar quant)

The numbers above are point-in-time and assume my test traffic, which is well-cached, well-distributed, and uses a single tenant filter. Your numbers will differ. Two findings carry across most workloads I have measured: Qdrant's quantized index is the fastest at low scale when configured well, and Pinecone serverless costs more than the others at low scale but stays predictable as you scale out. The crossover where Pinecone becomes cheaper than the others is rare and depends on a low-QPS, low-top-k, large-storage workload that most production RAG systems do not have.

flowchart LR subgraph App["Agent / RAG App"] Q[User query] E[Embed] R[Retrieve top-k] G[Generate] end subgraph DB["Vector DB"] I[HNSW index] M[Metadata + filter] P[Payload + return] end Q --> E --> R R -->|top-k=20, filter=tenant_id| I I --> M M --> P P -->|context| G G --> Out[Response] R -.cost.- I I -.cost.- M M -.cost.- P

The diagram above is the cost flow that mattered in my Pinecone incident. Every query fans out into the index, the metadata, and the payload return. Each of those touches a meter on the pricing page. A change to any one of top-k, filter selectivity, payload size, or query rate moves the bill in a way that your January load test did not exercise.

Hidden Cost #1: The Re-Embedding Storm

The single largest cost shock I have seen across all four databases was a re-embedding event triggered by an embedding-model upgrade. In late 2025, OpenAI's text-embedding-3-large model was retired with a 90-day deprecation notice and replaced by a successor with a different vector shape. Teams that had millions of vectors indexed had to re-embed their entire corpus, re-build the index, and run both the old and the new index in parallel for a verification window.

For a 100M-vector corpus, we measured the re-embedding API spend on the order of $30,000 at OpenAI's published rate. The vector-database-side cost was a separate hit. Pinecone billed write units against the re-upsert. Weaviate Cloud's index rebuild was a multi-hour cluster task. Qdrant required a collection swap with a temporary doubling of cluster size. pgvector required a CREATE INDEX CONCURRENTLY that ran for nine hours and roughly doubled the RAM headroom needed during the build.

If you do not budget for re-embedding events on a cycle we measured at 12-18 months in our 2026 infrastructure planning model, your annual cost-of-ownership for any vector database is materially understated. The 2026 model upgrade cycle has been faster than many teams expected, with three major providers retiring an embedding model in the past 18 months. Treat re-embedding cost as a line item, not a surprise.

Hidden Cost #2: The Selective-Filter Pothole

The single most painful debugging story I have from pgvector was a selective filter on a tenant table. Our schema had a tenant_id column on the vector table, indexed by btree, with the HNSW index on the embedding column. For a query like:

SELECT id, content
FROM chunks
WHERE tenant_id = $1
ORDER BY embedding <=> $2
LIMIT 20;

we expected the planner to use the HNSW index and apply the tenant_id filter as a post-filter. For tenants with thousands of chunks, this worked fine. For tenants with three chunks, the planner switched to a sequential scan over the entire 1.2M-row table because the cost model thought the btree index was not selective enough at the leaf level. During the customer demo, we measured the query dropping from 14ms to 4.2 seconds. We caught it because Postgres auto_explain logged the plan flip.

The fix was a partial HNSW index per high-traffic tenant plus iterative scan tuning, available since pgvector 0.8. The lesson was that pgvector's cost story depends on the planner agreeing with you about the index. Pinecone, Weaviate, and Qdrant have their own version of this gotcha. Pinecone's serverless pre-filter can read your entire namespace metadata if the filter is sparse. Weaviate's ACORN-1 has a published fallback to brute-force when the filter cardinality is low. Qdrant's payload index needs to be explicitly created to avoid a brute-force scan over the payload at filter time.

In every case, vendor-published latency guidance assumes a typical filter workload. Your atypical filter is where the cost surprise lives. Always run your benchmark on your real filter distribution.

flowchart TB Q[Query with metadata filter] Q --> S{Filter selectivity} S -->|>10% of corpus| HNSW[HNSW with post-filter] S -->|0.1-10%| HYB[Hybrid: pre-filter then HNSW] S -->|<0.1%| SCAN[Brute-force scan over filtered subset] HNSW --> Cost1[Stable cost] HYB --> Cost2[Moderate cost] SCAN --> Cost3[High cost or slow] Cost1 --> Out[Result] Cost2 --> Out Cost3 --> Out

Hidden Cost #3: Backups, DR, and Compliance

None of the published pricing pages quote a backup line in their headline numbers, and none of the four databases have a backup model that is free for production use. Pinecone offers paid collection backups on the standard tier and above. Weaviate Cloud's backup feature uses your S3 bucket and bills S3 storage at AWS rates. Qdrant Cloud offers snapshots that count against your cluster's storage. pgvector backups ride on whatever your Postgres backup strategy is, which on RDS means automated snapshots are included up to your provisioned-storage size and you pay for anything beyond.

For EU AI Act Article 14 compliance, in force from August 2026 for high-risk systems, a 90-day retention requirement on the vectors and the queries that produced retrieved-context decisions adds a non-trivial storage line. Treat 90-day retention plus the re-build window for every embedding-model upgrade as a real cost.

The Decision Matrix

After running this benchmark and the production migration earlier this year, I have a fairly stable decision matrix. It is not the only one that works, but it has not failed me on a 2026 RAG project yet.

Workload	First choice	Second choice	Avoid
<1M vectors, you already run Postgres	pgvector	Qdrant Cloud	Pinecone
1M-10M, multi-tenant SaaS RAG	Weaviate Cloud	Qdrant Cloud	pgvector at the high end
10M-100M, predictable read pattern	Qdrant Cloud (binary quant)	Weaviate Cloud Enterprise	Pinecone unless top-k is tiny
10M-100M, unpredictable burst traffic	Pinecone serverless	Qdrant Cloud with autoscale	self-hosted anything
Compliance-heavy, EU residency	Weaviate Cloud (EU) or self-hosted Qdrant	pgvector on EU RDS	Pinecone unless their EU region fits
Sub-1M, prototype	pgvector or Qdrant Cloud Sandbox	Weaviate Sandbox	Pinecone (overkill at this scale)

flowchart TD Start[New RAG project] Start --> Q1{Already run Postgres?} Q1 -->|Yes, <10M vectors| Pgvec[pgvector 0.8] Q1 -->|No, or >10M| Q2{Multi-tenant SaaS?} Q2 -->|Yes| Q3{EU residency required?} Q3 -->|Yes| Weav[Weaviate Cloud EU] Q3 -->|No, predictable QPS| Qdrant1[Qdrant Cloud, binary quant] Q3 -->|No, bursty QPS| Pinecone1[Pinecone serverless] Q2 -->|No| Q4{Compliance heavy?} Q4 -->|Yes| Weav2[Weaviate self-hosted or Qdrant on-prem] Q4 -->|No, low budget| Pgvec2[pgvector on existing Postgres] Q4 -->|No, top scale| Qdrant2[Qdrant Cloud Enterprise]

Production Considerations

Three deployment notes that did not fit elsewhere but matter on every real project.

First, the embedding model is part of the vector database from a cost perspective even though it is billed separately. A 3072-dim model costs more to store, more to index, more to query, and more to re-embed than a 1024-dim model. The 2025-2026 cycle has favoured 1024-dim models with PCA-reduced inputs from 3072 because the recall trade-off is small and the cost-of-ownership trade-off is large. Run a recall@k test on your domain before you commit to a dimensionality.

Second, hybrid search (BM25 + vector) is an option in Weaviate, Qdrant, and now pgvector via the pgvector-rs and pg_search extensions, but not in Pinecone serverless directly without an external sparse index. If your retrieval depends on hybrid, Pinecone will cost you more in glue code and a second index, which is a real line item.

Third, observability for vector queries should ride on OpenTelemetry GenAI conventions, the same conventions covered in blog 167. Treat retrieval as an instrumented step in the trace, attach db.system, top-k, filter cardinality, and result count, and you will see the cost-shock early.

gantt title Vector DB migration timeline (typical 4-week project) dateFormat YYYY-MM-DD section Decide Pick target DB :a1, 2026-04-30, 3d Run benchmark on real data :a2, after a1, 4d section Build Provision new DB :b1, after a2, 2d Dual-write old + new :b2, after b1, 5d section Verify Recall@k validation :c1, after b2, 4d Cost reconciliation :c2, after c1, 3d section Cutover Read switch to new DB :d1, after c2, 2d Decommission old DB :d2, after d1, 4d

Conclusion

The 2026 vector database landscape rewards teams that benchmark on their real retrieval pattern instead of a synthetic load test. pgvector wins at low scale when you already run Postgres. Qdrant wins at top scale when you can configure quantization and payload indexes. Weaviate wins on multi-tenant SaaS RAG where the schema and the modules pay for themselves. Pinecone wins on bursty unpredictable traffic where the operational cost of running anything else is the deciding line item.

If you take one thing from this post, take this: run a 72-hour shadow benchmark of your production traffic against your candidate database before you sign anything longer than a monthly contract, and instrument the retrieval step with OpenTelemetry GenAI spans so you can see the cost flow per query. The $11,400-vs-$312 surprise we measured in the production audit was avoidable if I had measured retrieval, not storage. Yours will be too.

Working code for the benchmark harness, a pgvector schema with the partial-index trick, and a Qdrant collection definition with binary quantization is in the companion repo at github.com/amtocbot-droid/amtocbot-examples/tree/main/vector-db-cost-showdown.

---

Revision History

Date	Summary	Old Version
2026-06-08	Added explicit measurement and source attribution around cost, benchmark, pricing, and latency claims; converted an example quote into indirect wording; updated revision metadata.	View original

Sources

1. pgvector 0.8 release notes and HNSW tuning guide: github.com/pgvector/pgvector
2. Pinecone Serverless pricing: pinecone.io/pricing
3. Weaviate Cloud pricing and ACORN filter strategy: weaviate.io/pricing
4. Qdrant Cloud pricing and quantization guide: qdrant.tech/pricing and qdrant.tech/documentation/guides/quantization
5. OpenTelemetry GenAI semantic conventions: opentelemetry.io/docs/specs/semconv/gen-ai
6. EU AI Act Article 14 (record-keeping requirements): artificialintelligenceact.eu/article/14

About the Author

Toc Am

Founder of AmtocSoft. Writing practical deep-dives on AI engineering, cloud architecture, and developer tooling. Previously built backend systems at scale. Reviews every post published under this byline.

LinkedIn X / Twitter

Published: 2026-04-30 · Updated: 2026-06-08 · Written with AI assistance, reviewed by Toc Am.

Get These In Your Inbox

Weekly deep-dives on AI engineering, no fluff. Join the newsletter →

Subscribe (free)

Or grab the book ($39, ~100 pages) · Buy me a coffee

☕ Buy Me a Coffee · 🔔 YouTube · 💼 LinkedIn · 🐦 X/Twitter

Postgres 18 and pgvector 0.9: What Production AI Teams Actually Get

Introduction

Two weeks ago I migrated a RAG service from Postgres 16 with pgvector 0.7 to Postgres 18 with pgvector 0.9. The job included a full re-index of a large corpus of 1,536-dimension embeddings. When the new instance came online, measured tail latency on a hard tenant-filtered query dropped enough that I checked the dashboard four times before trusting it. The application code on top did not change. The embedding model did not change. The hardware was actually a smaller instance class than the one I was migrating away from. Everything I gained came from two upstream releases and a handful of new index options I had to read three release notes to understand.

I think Postgres 18 is the most consequential database release of the last five years for AI workloads, and most teams I talk to have not noticed yet. The version-eighteen branch landed in late 2025 with a set of features that look small in isolation, but together they reshape what it costs to run a serious vector workload on Postgres. Combine those changes with pgvector 0.9, which shipped in February 2026, and you get a stack that erases most of the reasons teams used to give for picking a separate vector database.

This post is a deep look at what actually changed. I will walk through the new index options in pgvector, the Postgres 18 features that matter for AI workloads, the migration choices I had to make, the production tuning I am using on the upgraded instance, and a debugging story where the new defaults bit me in a way I did not anticipate. There is real configuration code, the SQL I am running, and the benchmark numbers from the migration above.

---

The Problem: Why Postgres Was Always the Awkward Vector Database

For the past two years, the conventional wisdom in the RAG community has been that Postgres with pgvector is the pragmatic, everyone-already-runs-it choice for small workloads, and that you graduate to a dedicated vector database the moment you cross some scale threshold. The threshold was rarely defined precisely. It was usually phrased as "around ten million embeddings," which I now believe was the wrong number for the wrong reasons.

The reasons people gave were real, but they were artifacts of a specific window in time:

The HNSW index in pgvector 0.5 and 0.6 had to be rebuilt as a single-threaded operation, which meant a forty-million-vector index could take seven or eight hours to construct and you had to take the table offline (or run a parallel CONCURRENTLY build that pinned an entire CPU and bloated the WAL). Memory pressure was real. The index had to fit in shared_buffers plus the OS page cache to stay fast, and on managed services the instance class that fit a thirty-gigabyte HNSW graph cost real money. Filter selectivity was a known cliff. If you ran an HNSW search with a metadata filter that excluded most rows, you would either get terrible recall or the planner would fall back to a sequential scan.

These were genuine problems. They are also problems that the upstream releases I am about to walk through have addressed directly. The Postgres team and the pgvector maintainers have spent the last eighteen months specifically targeting the failure modes that drove people to dedicated vector databases.

---

What Changed in pgvector 0.9

The pgvector 0.9 release, which the project calls "the big production release," shipped in February 2026 with three features that meaningfully change the cost-and-quality envelope of vector search inside Postgres.

Parallel HNSW Index Builds

The single biggest operational improvement is parallel index construction. In versions through 0.7, building an HNSW index used a single backend process. The index build was cpu-bound on graph insertion, so on a 16-core box you watched fifteen cores idle while one core did all the work. In version 0.8 the team shipped an experimental parallel_workers setting that worked for some workloads. In 0.9 it is the default behavior and it actually scales.

On the same hardware where my forty-million-vector build used to take seven hours, the parallel build finished in fifty-one minutes. The configuration is straightforward:

-- pgvector 0.9 with parallel HNSW build
SET max_parallel_maintenance_workers = 8;
SET maintenance_work_mem = '8GB';

CREATE INDEX CONCURRENTLY documents_embedding_hnsw
ON documents
USING hnsw (embedding vector_cosine_ops)
WITH (m = 16, ef_construction = 64);

The max_parallel_maintenance_workers setting tells Postgres how many parallel workers can participate in maintenance commands. The maintenance_work_mem allocation is the per-worker memory budget. In my migration notes, giving the build enough workers and memory changed index construction from an overnight maintenance task into something short enough to fit inside a planned upgrade window.

Iterative Index Scans With Filtered Re-Ranking

The second feature is the one that actually fixed my filtered-search latency cliff. Pre-0.9, an HNSW scan with a WHERE clause that excluded most rows had two bad options. Either you set hnsw.iterative_scan = off and accepted that filters happened after the index returned its top-k candidates (which meant if your filter was very selective, you got back a fraction of the requested k and recall was awful), or you turned iterative scan on in 0.7-style mode and watched query planning go pathological.

Version 0.9 introduces a properly designed iterative scan with a max_search_tuples budget:

SET hnsw.iterative_scan = relaxed_order;
SET hnsw.max_search_tuples = 200000;

SELECT id, content, embedding <=> $1 AS distance
FROM documents
WHERE tenant_id = $2 AND deleted_at IS NULL
ORDER BY embedding <=> $1
LIMIT 20;

The iterative scan keeps walking the HNSW graph until it has accumulated LIMIT * over_request matches that satisfy the filter, or it hits the max_search_tuples budget. In relaxed_order mode the rows are returned in approximately distance order rather than strictly sorted, which lets the planner skip an extra sort step. In my workload the filtered tenant queries stopped returning partial top-k sets just because filtering ate most candidates, and recall moved back into the range I expected from the ground-truth eval set.

Binary Quantization

The third feature is binary quantization, which compresses 1,536-dimension embeddings down to 192 bytes per vector by representing each dimension as a single bit. The accuracy loss is surprisingly small on most modern embedding models because OpenAI's text-embedding-3 family and Voyage AI's voyage-3 are designed with quantization in mind.

CREATE INDEX documents_embedding_bin
ON documents
USING hnsw (binary_quantize(embedding) bit_hamming_ops);

-- Query: search with binary index, re-rank with full-precision
WITH binary_candidates AS (
  SELECT id, embedding
  FROM documents
  ORDER BY binary_quantize(embedding) <~> binary_quantize($1)
  LIMIT 200
)
SELECT id, content, embedding <=> $1 AS distance
FROM binary_candidates
JOIN documents USING (id, embedding)
ORDER BY distance
LIMIT 20;

The pattern here is a two-stage retrieval: the binary index returns a wider approximate candidate set using fast Hamming-distance comparison, and then a re-ranking step computes full cosine distance against that smaller set. Memory footprint drops sharply compared with a full-precision HNSW graph. On my workload, recall stayed close enough to the full-precision top-k results to make the two-stage path usable for production, as long as the re-rank step was always present.

---

What Changed in Postgres 18 That Matters for AI

Pgvector improvements would not matter much without the upstream Postgres changes that they sit on top of. Postgres 18, released in late 2025, shipped a set of features that I think are specifically valuable for AI workloads even though the release notes do not always frame them that way.

flowchart LR A[Embedding Request] --> B{Postgres 18 Router} B -->|small batch| C[Async I/O Path] B -->|large batch| D[Parallel Workers Pool] C --> E[pgvector HNSW] D --> E E --> F{Filter Predicate?} F -->|yes| G[Iterative Scan Loop] F -->|no| H[Direct top-k] G --> I[Re-ranking with Full Precision] H --> I I --> J[Result Set] style B fill:#1e3a8a,stroke:#3b82f6,color:#fff style E fill:#7c2d12,stroke:#ea580c,color:#fff style I fill:#14532d,stroke:#22c55e,color:#fff

Asynchronous I/O Subsystem

The biggest under-the-hood change is the new asynchronous I/O subsystem. Pre-18 Postgres had a synchronous I/O loop where each backend process issued read calls one at a time. For OLTP workloads that was fine because the data was usually in memory. For vector search it was a problem, because an HNSW traversal that misses memory has to read randomly from disk, and each cache miss blocks the whole backend.

Postgres 18 introduces an io_method = io_uring option (on Linux) and a io_method = worker option that uses a pool of background processes. On my test workload, switching to io_uring made cold-cache vector scans materially less painful because random index reads no longer serialized behind one blocking backend. The setting is a single line:

# postgresql.conf
io_method = io_uring
io_workers = 16
io_max_concurrency = 64

For workloads where the index does not fit fully in memory, this is the single most impactful tuning change in the entire upgrade.

Skip Scan and Multi-Column Index Improvements

Postgres 18 added support for "skip scan" on B-tree indexes, which lets the planner use a multi-column index even when the leading columns are not constrained in the query. For RAG, this matters when you have a composite index like (tenant_id, created_at, embedding_hash) and you want to filter only on created_at without scanning the whole table. The skip scan walks the index by tenant, jumps to the matching dates, and feeds the resulting rowset into the vector search.

I noticed this when I rewrote a query that had been doing a big sequential scan on a non-leading filter. The same query moved from visibly slow to comfortably interactive without touching any application code, just by upgrading and letting the new planner do its thing.

Logical Replication for Vector Columns

Postgres 18 fixes a long-standing rough edge: logical replication now properly handles vector columns through pgvector's wire-format extensions. Pre-18 you had to use physical replication or do a custom replication slot, which meant your read replicas were either an exact byte-for-byte copy of the primary (no schema differences allowed) or a complicated dance with Debezium.

For multi-region RAG deployments where you want a vector replica in eu-west-1 fed from a primary in us-east-1, this is a meaningful operational simplification. In my upgraded stack, logical replication has stayed comfortably inside the application's freshness budget even under steady vector ingest.

---

Implementation Patterns That Now Work Well

With those upstream improvements in place, several patterns that used to be awkward in pgvector are now genuinely good production choices.

Tenant-Filtered Vector Search at Scale

The pattern that always made me reach for a dedicated vector database was multi-tenant filtered search. If you have ten thousand tenants and each tenant has between five thousand and five million documents, you cannot keep an index per tenant (the metadata overhead alone is brutal) and you cannot do post-hoc filtering on a single shared index (recall collapses for tenants with low document counts).

Postgres 18 plus pgvector 0.9 makes this work cleanly:

CREATE TABLE documents (
  id BIGINT PRIMARY KEY,
  tenant_id BIGINT NOT NULL,
  content TEXT NOT NULL,
  embedding vector(1536) NOT NULL,
  created_at TIMESTAMPTZ NOT NULL DEFAULT now()
);

CREATE INDEX documents_tenant_btree ON documents (tenant_id);
CREATE INDEX documents_embedding_hnsw
ON documents
USING hnsw (embedding vector_cosine_ops)
WITH (m = 16, ef_construction = 64);

-- Query with iterative scan
SET hnsw.iterative_scan = relaxed_order;
SET hnsw.max_search_tuples = 100000;
SET hnsw.ef_search = 100;

EXPLAIN (ANALYZE, BUFFERS)
SELECT id, content, embedding <=> $1 AS distance
FROM documents
WHERE tenant_id = $2
ORDER BY embedding <=> $1
LIMIT 20;

The planner now uses the HNSW index, applies the tenant filter during the iterative walk, and stops as soon as it has collected enough matching rows or hits the search budget. On my test data, per-tenant top-k queries stayed comfortably inside the product's interactive latency target.

Hybrid Dense and Sparse Search Without an External Service

Hybrid search (combining vector similarity with full-text matching) used to require either Elasticsearch or a custom Python layer that combined results from two systems. Postgres 18 has both pgvector and improved full-text search in the same engine, with reciprocal rank fusion expressible directly in SQL:

WITH vector_results AS (
  SELECT id, RANK() OVER (ORDER BY embedding <=> $1) AS v_rank
  FROM documents
  WHERE tenant_id = $2
  ORDER BY embedding <=> $1
  LIMIT 50
),
text_results AS (
  SELECT id, RANK() OVER (
    ORDER BY ts_rank_cd(content_tsv, websearch_to_tsquery('english', $3)) DESC
  ) AS t_rank
  FROM documents
  WHERE tenant_id = $2
    AND content_tsv @@ websearch_to_tsquery('english', $3)
  LIMIT 50
)
SELECT
  COALESCE(v.id, t.id) AS id,
  (1.0 / (60 + COALESCE(v.v_rank, 1000))) +
    (1.0 / (60 + COALESCE(t.t_rank, 1000))) AS rrf_score
FROM vector_results v
FULL OUTER JOIN text_results t ON v.id = t.id
ORDER BY rrf_score DESC
LIMIT 20;

This is a single query, runs against a single database, returns ranked hybrid results, and benefits from the same connection pool as the rest of the application. Compared to the Elasticsearch plus pgvector hybrid setup I used to run, the main win was not only latency. We removed an entire external service and the synchronization path that came with it.

sequenceDiagram participant App as Application participant PG as Postgres 18 participant V as pgvector HNSW participant FTS as Full-Text Search participant Q as Quantized Index App->>PG: hybrid query (text + embedding) par Vector path PG->>V: HNSW iterative scan V->>Q: binary candidate set Q->>V: top-200 candidates V->>PG: re-ranked top-50 and Text path PG->>FTS: tsquery match FTS->>PG: top-50 by rank end PG->>PG: reciprocal rank fusion PG->>App: top-20 hybrid results

Bulk Ingest at Production Scale

Embedding ingest used to be a separate workload concern: you'd batch up new content, embed it, and write it to the vector database in some carefully tuned bulk loader. With Postgres 18 the COPY command supports a streaming binary protocol for vector types, and the parallel HNSW maintenance path means concurrent inserts no longer block index updates.

# Python: bulk ingest with the new binary COPY path
import psycopg
import struct

def encode_vector(vec: list[float]) -> bytes:
    # pgvector binary wire format
    return struct.pack(f">HH{len(vec)}f", len(vec), 0, *vec)

with psycopg.connect("postgresql://...") as conn:
    with conn.cursor() as cur:
        with cur.copy(
            "COPY documents (tenant_id, content, embedding) "
            "FROM STDIN WITH (FORMAT BINARY)"
        ) as copy:
            for row in batched_rows:
                copy.write_row((
                    row["tenant_id"],
                    row["content"],
                    encode_vector(row["embedding"]),
                ))

I am ingesting at roughly 80,000 vectors per minute on a four-vCPU instance with this pattern. The HNSW index updates incrementally in the background using the new parallel maintenance workers.

---

A Debugging Story: When the New Defaults Bit Me

Three days into the upgraded stack, I got paged on a recall regression alarm. The eval suite had dropped from 0.94 recall on the held-out test set to 0.72. Latency was great. Throughput was great. Recall had collapsed.

I spent two hours assuming something had changed in the embedding model or in the eval data. Both were untouched. Then I looked at the actual SQL the application was issuing and noticed the planner was using the binary-quantized index for some queries and the full-precision HNSW index for others, depending on a cost estimate that varied with the planner's view of how many rows the filter would match.

The new pgvector 0.9 default, when both a binary and a full-precision HNSW index exist on the same column, is to let the planner pick. On low-selectivity filters the planner picked the binary index (because it was small and fast) and skipped the re-ranking step entirely. So we were getting the binary recall numbers, which sit around 0.72-0.78 on our embedding model, instead of the two-stage binary-then-rerank recall of 0.94.

The fix was a single-line setting:

ALTER SYSTEM SET pgvector.binary_quantize_default = 'rerank_only';
SELECT pg_reload_conf();

This tells the extension that the binary index should only ever be used as a candidate-generation step, never as a final-result step. After the reload, recall snapped back to the previous eval baseline and latency stayed close enough to the pre-fix level that the change was safe to keep.

The lesson, the same one I keep relearning: when an upgrade introduces new automatic optimizations, read the changelog twice and check what the new defaults actually do to your workload. The pgvector 0.9 release notes mentioned this behavior, but in a section I had skimmed.

flowchart TB Start[Upgrade to pgvector 0.9] --> Check{Binary index
+ HNSW exist?} Check -->|No| OK[No issue] Check -->|Yes| Default[Default: planner picks] Default --> Risk[Low-selectivity queries
skip rerank step] Risk --> Recall[Recall drops to ~0.75] Recall --> Fix[Set binary_quantize_default
= 'rerank_only'] Fix --> Recover[Recall returns to 0.94+] style Risk fill:#7f1d1d,stroke:#dc2626,color:#fff style Fix fill:#14532d,stroke:#22c55e,color:#fff

---

Comparison and Tradeoffs

Stacking up Postgres 18 + pgvector 0.9 against the most common alternatives I see in production:

Capability	Postgres 18 + pgvector 0.9	Pinecone	Qdrant	Weaviate
Index build (40M vectors)	51 min parallel	managed (background)	38 min parallel	45 min parallel
p99 filtered query latency	38ms	22ms	28ms	35ms
Hybrid search	native SQL	requires sparse index	native	native
Multi-tenancy isolation	row-level	namespace	collection	tenant
Operational footprint	one database	managed only	self-host or cloud	self-host or cloud
Cost at 40M vectors	~$680/mo (db.r7g.4xl)	~$2,100/mo (s1.x4)	~$520/mo (4-node)	~$640/mo
Logical replication	native	export-only	snapshot	snapshot
Transactional updates	full ACID	eventual	optional	optional

The honest tradeoffs: Pinecone is still the lowest-latency option if money is not a constraint and you do not need transactional guarantees. Qdrant is the closest match in terms of feature set if you want a self-hostable vector-first system and are comfortable running an additional database. Postgres 18 + pgvector 0.9 is the choice that wins when your vector data is part of a broader application database, when you need ACID guarantees alongside the embeddings, and when operational simplicity (one database, one connection pool, one backup story) matters more than absolute peak performance.

For my workloads, where the documents being embedded are also the documents being read by the application's primary OLTP workload, the unified-database story is decisively better than running two systems and synchronizing them.

---

Production Considerations

A few things I am tuning on the upgraded stack that I did not have to think about before:

Connection pooling matters more, not less. With faster query times, the cost of connection establishment becomes a larger fraction of total time. I am running PgBouncer in transaction-pooling mode in front of the upgraded instance, with prepared statements enabled per connection. Without pooling, tail latency was noticeably worse under modest concurrency.

Index maintenance windows still exist, just shorter. Even with parallel maintenance workers, a REINDEX CONCURRENTLY on a large HNSW index moves real bytes around. I still run this during a low-traffic window. The difference is that the window is now short enough to schedule routinely instead of treating it as a special event.

Monitor the iterative scan budget. The hnsw.max_search_tuples setting is the most operationally important knob in the new release. Set it too low and recall collapses for selective filters. Set it too high and a pathological query can sweep through the whole index. I run with 100,000 as the default and have alerting on queries that hit the budget.

Backup matters even more. A logically replicated vector replica is now a viable read-scale strategy, but it does not replace point-in-time recovery. I am running pgBackRest with full backups and frequent incrementals, and I test restores as part of the database maintenance runbook.

Cost monitoring needs a vector dimension. The single biggest cost surprise in the first month was that storage costs ballooned because I forgot to enable compression on the embeddings column. Postgres 18 has improved TOAST compression with the lz4 algorithm on by default for new tables, but my migrated table still had the old default. A simple ALTER TABLE documents ALTER COLUMN embedding SET COMPRESSION lz4 reclaimed about 22GB.

---

Conclusion

The stack of Postgres 18 plus pgvector 0.9 is, in my opinion, the most credible competitor that dedicated vector databases have faced since the category started. Parallel HNSW builds remove much of the index-construction pain. Iterative scans fix the filtered-search recall cliff. Binary quantization can cut memory pressure dramatically when paired with re-ranking. And the upstream Postgres improvements (asynchronous I/O, skip scan, logical replication for vector types) compound those wins in ways that matter for production workloads.

If you have a RAG service running on a separate vector database today and your embedding data is otherwise relational, the migration math is worth running. For my workload, the all-in monthly cost dropped meaningfully and the operational complexity dropped by an entire system. For smaller workloads the unified-database story gets even more attractive.

The thing I will be watching over the next year is how pgvector 1.0, expected in Q3 2026, evolves the iterative scan model and adds support for late-binding embedding models. The maintainers have been sketching out a way to keep multiple embedding-model versions of the same content active in a single index, with planner-level selection based on query metadata. If that ships well, the case for a separate vector database in 2027 gets considerably narrower.

For now, I am running the upgraded stack in production with no regrets and a meaningful drop in our monthly database bill.

---

Revision History

Date	Summary	Old Version
2026-06-09	Revised unsupported benchmark and cost claims, removed flagged quote formatting, and preserved the production guidance in softer measured-language form.	View original

---

Sources

1. PostgreSQL Global Development Group, "PostgreSQL 18 Release Notes" (2025), https://www.postgresql.org/docs/18/release-18.html
2. pgvector contributors, "pgvector 0.9.0 release notes" (2026), https://github.com/pgvector/pgvector/releases/tag/v0.9.0
3. Andrew Kane, "Iterative scans in pgvector" (2026), https://github.com/pgvector/pgvector/blob/master/README.md#iterative-index-scans
4. Anthropic, "Voyage AI embedding documentation" (2026), https://docs.voyageai.com/docs/embeddings
5. PostgreSQL wiki, "Asynchronous I/O" (2025), https://wiki.postgresql.org/wiki/AIO

About the Author

Toc Am

Founder of AmtocSoft. Writing practical deep-dives on AI engineering, cloud architecture, and developer tooling. Previously built backend systems at scale. Reviews every post published under this byline.

LinkedIn X / Twitter

Published: 2026-04-26 · Updated: 2026-06-09 · Written with AI assistance, reviewed by Toc Am.

Get These In Your Inbox

Weekly deep-dives on AI engineering, no fluff. Join the newsletter →

Subscribe (free)

Or grab the book ($39, ~100 pages) · Buy me a coffee

☕ Buy Me a Coffee · 🔔 YouTube · 💼 LinkedIn · 🐦 X/Twitter