Choosing a vector database without regretting it in six months

What the regret actually looks like

Nobody regrets a vector database because it was slow. The regret arrives in a different shape, usually about two quarters in, and it sounds like one of these four sentences.

"Search works fine until we filter by clearance level, and then it returns nothing." "It cannot find document numbers, only topics." "Our monthly bill tripled and we did not add users." "We need to change the embedding model and re-indexing means a two-week outage nobody approved."

Every one is a decision made during a two-day proof of concept, when the corpus was ten thousand chunks, there were no permissions, and every query was a paraphrased question typed by the engineer who built it. Benchmarks at that scale measure almost nothing that matters later. Approximate nearest-neighbor search is well understood and the mainstream products are all competent at it. The dimensions below are where they genuinely differ, and each is checkable against vendor documentation before you commit.

Filtering: the dimension that decides most projects

Almost every real system filters. By tenant, by clearance, by date range, by document type, by whether the record is superseded. Vector similarity and metadata filtering are structurally awkward to combine, and how a product resolves that awkwardness is the single most consequential difference between the options.

The naive approach is post-filtering: run the approximate search, get your top k, then discard whatever fails the filter. The pgvector documentation states the consequence plainly — with approximate indexes, "queries with filtering can return less results since filtering is applied after the index is scanned" — and supplies an example worth memorizing: "If a condition matches 10% of rows, with HNSW and the default hnsw.ef_search of 40, only 4 rows will match on average."

Read that again with a permission filter in mind. The chunk exists. The user is allowed to see it. The system returns nothing, and the user concludes the document is not in the corpus. That is not a ranking bug a better model tunes away.

Each product has a different answer, and the answers are not equivalent:

• pgvector added iterative index scans in 0.8.0, which per the documentation "will automatically scan more of the index until enough results are found (or it reaches hnsw.max_scan_tuples or ivfflat.max_probes)," in strict or relaxed ordering mode.
• Qdrant takes a structural approach. Its indexing documentation describes filterable HNSW as "extending the HNSW graph with additional edges based on indexed payload values," so filters can be applied during graph traversal rather than after it. It also documents a full_scan_threshold_kb below which the planner simply scans everything, and an ACORN mode that "explores not just direct neighbors (first hop), but also neighbors of neighbors (second hop) when direct neighbors are filtered out."
• Pinecone describes single-stage filtering that merges the vector and metadata indexes into one index, and characterizes post-filtering as carrying "a high latency penalty for retrieving more items than needed" with "no guarantee the result set would include all the items you actually want."

A scheduling trap sits inside the Qdrant approach, and it generalizes. Its documentation is explicit: "For the HNSW graph to be optimized for filtered search, it's highly recommended to create all payload indices immediately after collection creation, before ingesting data." Decide your filter fields before you load a hundred million vectors. This is ordinary database index design discipline, skipped for the ordinary reason — it feels premature until it is expensive.

The test to run: apply your most selective realistic filter to a corpus at the scale you expect in a year, request 10 results, and count what comes back. Fewer than 10 when you know more than 10 matching records exist means you found the problem before it found you.

Hybrid search: where dense vectors quietly fail

Dense embeddings are built to place semantically similar text near each other — precisely the wrong behavior for identifiers. Part number 4471-B and 4471-D differ by one character and mean different objects; statute cites, form numbers, error codes, and invoice numbers all have this property. A vector index will cheerfully hand you the neighbor.

The fix is a lexical channel alongside the dense one, with the two ranked lists fused. The standard method is reciprocal rank fusion, from Cormack, Clarke and Buettcher (SIGIR 2009), a paper whose title states the finding plainly: reciprocal rank fusion outperforms Condorcet and individual rank learning methods.

What varies is whether the database does this for you. Weaviate's documentation describes hybrid search as combining "the results of a vector search and a keyword (BM25F) search by fusing the two result sets," offers both Ranked Fusion and Relative Score Fusion, notes that "Relative Score Fusion is the default fusion method starting in v1.24," and exposes an alpha parameter where "an alpha of 1 is a pure vector search" and "an alpha of 0 is a pure keyword search." OpenSearch and Elasticsearch, being search engines that gained vector types, bring mature BM25 with them. pgvector sits inside PostgreSQL, so the lexical half is a tsvector index and the fusion is SQL you write and own.

None of these is wrong. But "we will add keyword search later" is a far larger project against a store with no lexical side, and that asymmetry belongs in the decision rather than next year's backlog.

Index type and the memory bill

Index choice is where cost becomes arithmetic rather than opinion. The pgvector documentation gives the clean summary of the two dominant families. HNSW "creates a multilayer graph" and "has better query performance than IVFFlat (in terms of speed-recall tradeoff), but has slower build times and uses more memory." IVFFlat "divides vectors into lists, and then searches a subset of those lists that are closest to the query vector," with "faster build times" and "less memory than HNSW, but lower query performance."

Milvus's index documentation adds a useful nuance for anyone retrieving large candidate sets: "Graph-based index types usually outperform IVF variants in terms of QPS," but "IVF variants particularly fit in the scenarios with a large topK (for example, over 2,000)." It also notes that disk-backed approaches such as DiskANN help with large datasets but introduce "potential IOPS bottlenecks" — you traded RAM for storage latency, which is real and not free.

Now the number that decides instance size. OpenSearch's k-NN documentation estimates HNSW memory as 1.1 * (4 * dimension + 8 * M) bytes per vector, where M is the number of bidirectional links per graph node. An illustrative case: ten million chunks at 1,024 dimensions with M = 16 gives 1.1 × (4,096 + 128) = about 4,646 bytes per vector, or roughly 46 GB of index. Swap to a 384-dimension embedding model and the same formula gives about 1,830 bytes per vector, roughly 18 GB — about two and a half times cheaper, before any quantization.

That reframes the embedding decision. Dimension is not only a quality knob; it is a budget line that scales linearly with corpus size, and a larger model has to earn that difference on your evaluation set.

Quantization is the other lever, and the published trade-offs are consistent. Milvus documents SQ8 as "reducing memory usage by 75% compared to 32-bit floats," and product quantization as achieving "higher compression ratios (e.g., 4-32x) at the cost of marginally reduced recall," noting that "PQ typically offers a better recall rate at similar compression rates when compared to SQ." Compression is close to free when a reranker re-scores the candidates anyway — you only need the approximate index to get the right chunk into the top 100.

Operational burden, which nobody scopes

Ask a plain question about each candidate: who is on call at 2 a.m., and what do they do?

An extension inside a database you already run is a different proposition from a new distributed system. If PostgreSQL is already in production, pgvector inherits your backups, replication, access control, monitoring, and on-call rotation. That inheritance is worth more than a benchmark chart, and it is why the option you already operate wins more often than infrastructure discourse suggests.

A dedicated vector database earns its operational cost when scale, filtering behavior, or index features genuinely require it — answerable questions, not matters of taste. A managed service moves the burden to a vendor and moves your data across a boundary: a straightforward trade commercially, a much harder one when the corpus is regulated. Work that constraint before the technical comparison, not after — our guides on regulated data and air-gapped deployment cover the shape of it.

Three operational costs are routinely missed. Index builds are not free — pgvector's own note that HNSW "has slower build times" becomes a maintenance window when you rebuild over hundreds of millions of vectors. Version behavior changes: Weaviate's default fusion method changed at v1.24, a sensible improvement that also shifts result ordering across an upgrade if nobody reads the release notes. And ingest has to be continuous, because a retrieval system that lags the source of truth by a month produces confidently outdated answers, which is a leading cause of confident wrong answers.

What actually locks you in

The query API is the least of it. Every mainstream store does upsert, query-with-filter, and delete; a typed interface makes them substitutable in a few hundred lines. The real dependencies are elsewhere.

The embedding model. Vectors from two models are not comparable, so changing models means re-embedding and re-indexing the whole corpus. That is compute time, wall-clock time, and a dual-write or shadow-index plan if you cannot take an outage — the migration cost people discover late, and it is unrelated to which database you picked.

The metadata schema. Products publish hard constraints that shape your design. Pinecone's indexing documentation states that "Pinecone supports 40KB of metadata per record" and that "each $in or $nin operator accepts a maximum of 10,000 values." Those are reasonable limits, and a permission model that assumes unlimited metadata turns them into a rewrite. pgvector publishes ceilings too, and the ones that bind are not the ones usually quoted. The vector type stores up to 16,000 dimensions, but an index can only be built on up to 2,000; halfvec raises the index ceiling to 4,000 and binary quantization to 64,000, while sparsevec indexes cap at 1,000 non-zero elements. A 3,072-dimension embedding therefore fits the column and not the index — a surprise much better had in week one than after ingest. Read each candidate's constraints before the schema hardens.

The evaluation harness. This is the asset that makes migration survivable. Without a labeled set of real queries mapped to the passages that answer them, moving stores is faith-based: you cannot show that quality held. With one, migration is a measurable exercise. Build it in week one — it costs a half-day of subject-matter-expert time and it makes every later decision testable.

A one-week decision procedure

Five steps, in order. The order matters.

1. Write down the boundary constraint first. Where can this data legally and contractually live? That question eliminates candidates faster than any benchmark, and running it last wastes the evaluation.

2. Build a gold set of 50 to 100 real queries from real users, each paired with the passage a knowledgeable colleague says answers it. Include the identifier lookups — part numbers, cite numbers — because those are the queries dense-only retrieval fails.

3. Test filtering at realistic scale and selectivity. Load enough vectors that approximation matters, apply your tightest real filter, and count returned results against known matches. This test separates candidates more sharply than latency ever will.

4. Compute the memory bill from the published formula at your projected corpus size, for two embedding dimensions, with and without quantization. Then check whether the larger model wins on your gold set. Often it does not.

5. Ask who operates it. If the answer is "the same team that already runs our PostgreSQL," that is a strong argument. If it is "we will hire for that," price the hiring.

Notice what is absent: a leaderboard of queries per second. Choosing between mainstream products on raw throughput optimizes the dimension least likely to be the thing you regret. The same logic applies a level up, where the first question is whether retrieval is the right architecture at all — our guides on RAG versus extraction and build versus buy cover that layer.

Sources: pgvector documentation — index types, iterative index scans, filtering behavior, vector type and index dimension limits; Qdrant — Indexing (filterable HNSW, payload indexes, full-scan threshold, ACORN); Weaviate — Hybrid search (fusion methods and alpha); Pinecone — Indexing overview (metadata limits, namespaces); Pinecone — The Missing WHERE Clause in Vector Search (single-stage filtering); OpenSearch — k-NN index memory estimation; Milvus — Index Explained (index families, quantization trade-offs, DiskANN); Cormack, Clarke and Buettcher, "Reciprocal Rank Fusion Outperforms Condorcet and Individual Rank Learning Methods" (SIGIR 2009). Product documentation changes; verify current behavior against the vendor's docs before you commit. Analysis and framing by Precision AI Academy.

Common questions

Does the choice of vector database matter as much as people say? Less than the marketing suggests and more than skeptics claim. Raw nearest-neighbor speed is close to a commodity. What differs materially is how filtering interacts with the index, whether keyword and vector search can be fused inside the engine, which index and quantization options exist, and how much operational work the thing demands. Those are the dimensions worth comparing.

Why do filtered vector searches return too few results? Because in some systems the filter is applied after the approximate index has already been scanned. The pgvector documentation states that with approximate indexes, filtering "is applied after the index is scanned," and gives the example that if a condition matches 10% of rows, with HNSW at the default ef_search of 40, only about 4 rows will match on average. pgvector 0.8.0 added iterative index scans that keep scanning until enough results are found.

Do I need hybrid search, or is vector search enough? If your corpus contains identifiers people actually type — part numbers, statute cites, error codes — dense vectors alone will blur near-identical strings together. Weaviate documents hybrid search as fusing vector and BM25F result sets, with an alpha parameter where 1 is pure vector and 0 is pure keyword. Reciprocal rank fusion, from Cormack, Clarke and Buettcher (SIGIR 2009), is the standard way to combine two ranked lists.

What actually locks you into a vector database? Rarely the query API, which is a thin adapter. The expensive dependencies are the embedding model whose vectors fill the index, the chunking scheme, the metadata schema shaped around one product's constraints, and the evaluation harness that proves quality. Changing embedding models means re-embedding and re-indexing the entire corpus.