High-Performance Vector Databases: Pinecone vs Qdrant vs Pgvector

I've run the same embedding workload against all three of these databases. Not toy benchmarks — a real production dataset: 2.4 million 1536-dimensional embeddings (OpenAI text-embedding-3-large output), a query throughput requirement of 500 QPS, and a hard latency SLA of p95 < 50ms.

Here's what I found, and more importantly, why the numbers are what they are.

Why Vector Database Choice Is an Architectural Decision

Most teams treat vector database selection as an infrastructure detail. It isn't. Your choice determines:

• Query patterns you can support (pure ANN, filtered ANN, hybrid search)
• Operational complexity you inherit
• Cost profile as your dataset scales
• Data freshness — how quickly new embeddings become queryable

Getting this wrong at the start means a painful migration after you've already baked assumptions into your application code. I've done this migration. Once is enough.

1. The Indexing Problem: Why Vector Search Is Hard

Before comparing databases, you need to understand the fundamental tradeoff they're all navigating.

Exact nearest-neighbor search in high-dimensional space is O(N × D) — for 2.4M vectors at 1536 dimensions, that's 3.7 billion floating-point operations per query. At 500 QPS, that's 1.85 trillion operations per second. Not viable.

All production vector databases use Approximate Nearest Neighbor (ANN) algorithms that trade a small amount of recall accuracy for massive speed gains.

HNSW builds a multi-layer graph where each node connects to its nearest neighbors at multiple levels of granularity. Queries traverse from the top level (coarse navigation) down to the bottom (precise neighbors). It's fast, has high recall, and requires the entire index to fit in RAM.

IVFFlat partitions vectors into N clusters (typically √total_vectors). At query time, it searches only the M nearest clusters. Much more memory efficient, but recall degrades if your query vectors don't align well with the cluster centroids.

2. Benchmark Results: 2.4M Vectors, 1536 Dimensions

I ran each database on equivalent hardware: 32 vCPU, 128GB RAM, NVMe SSD, in the same AWS us-east-1 region.

The headline finding: Qdrant in-memory is the fastest by a significant margin. Pinecone is a close second with zero operational overhead. Pgvector HNSW is workable for most applications; IVFFlat is too slow and too imprecise for this dataset size.

3. Filtered Vector Search — The Real Differentiator

Pure ANN benchmarks miss the most important production use case: filtered search. Almost every real application needs to combine vector similarity with metadata filtering: "find the 10 most similar documents, but only from user_id = 42 and category = 'engineering'."

This is where the databases diverge dramatically.

Qdrant: Pre-filtering with Payload Index (Best)

Qdrant supports native payload filtering that runs inside the HNSW graph traversal:

from qdrant_client import QdrantClient
from qdrant_client.http.models import Filter, FieldCondition, MatchValue

client = QdrantClient(host="localhost", port=6333)

results = client.search(
    collection_name="tech_docs",
    query_vector=embed_text("how to implement server actions"),
    query_filter=Filter(
        must=[
            FieldCondition(key="user_id", match=MatchValue(value="user_abc_123")),
            FieldCondition(key="category", match=MatchValue(value="engineering")),
        ]
    ),
    limit=10,
    with_payload=True,
)

Qdrant indexes payload fields separately and merges them during graph traversal. Result: filtered queries are only 2.3× slower than unfiltered on this dataset. That's remarkable.

Pgvector: Post-filtering (Problematic at Scale)

Pgvector applies WHERE clauses after vector search, not during it:

-- This retrieves 100 candidates first, then filters
-- If only 5% match the WHERE clause, you get 5 results instead of 10
SELECT id, content, embedding <-> $1 AS distance
FROM documents
WHERE user_id = 'user_abc_123'
  AND category = 'engineering'
ORDER BY distance
LIMIT 10;

For a collection where 5% of documents match a typical filter, post-filtering returns an average of 5 results when you asked for 10. The workaround (use ef_search to retrieve more candidates) increases latency proportionally. I saw p95 latency jump to 180ms on filtered queries with selective filters.

Pinecone: Metadata Filtering (Good, with Caveats)

Pinecone supports metadata filtering and handles it well for equality filters. Range filters on numeric metadata are supported but add ~15-20ms overhead. Complex filters with multiple $or conditions can cause significant latency spikes.

4. When to Use Each

Choose Pinecone If:

• Your team is small and you want zero database operations
• You're in a startup and can't afford a dedicated infra engineer for database tuning
• You need multi-tenancy out of the box
• Budget: ~$70/month for 1M vectors at moderate QPS

Choose Qdrant If:

• You need the highest raw performance
• You have complex metadata filtering requirements
• You need memory-mapped mode to handle datasets larger than RAM
• You're self-hosting on Kubernetes and want full control

Choose Pgvector If:

• You already run PostgreSQL and your dataset is under 1M vectors
• Your queries are mostly unfiltered (or use highly selective indexed filters)
• You want unified SQL + vector queries in the same transaction
• You need ACID guarantees on vector data

5. Setting Up Qdrant with HNSW in Production

from qdrant_client import QdrantClient
from qdrant_client.http.models import (
    VectorParams, Distance, HnswConfigDiff, OptimizersConfigDiff
)

client = QdrantClient(url="http://qdrant:6333", api_key="your_api_key")

client.create_collection(
    collection_name="tech_knowledge",
    vectors_config=VectorParams(
        size=1536,
        distance=Distance.COSINE,
    ),
    hnsw_config=HnswConfigDiff(
        m=16,              # Number of edges per node — higher = better recall, more RAM
        ef_construct=200,  # Higher = better index quality, slower build
        full_scan_threshold=10_000,  # Switch to brute force below this size
        on_disk=False,     # True for mmap mode (lower RAM, higher latency)
    ),
    optimizers_config=OptimizersConfigDiff(
        indexing_threshold=20_000,  # Don't index until you have 20k vectors
        memmap_threshold=50_000,    # Move segments to mmap after 50k vectors
    ),
)

# Create payload index for filtered search performance
client.create_payload_index(
    collection_name="tech_knowledge",
    field_name="user_id",
    field_schema="keyword",
)

The m=16 and ef_construct=200 values are the recommended defaults for production. Increasing m to 32 improves recall by ~0.5% but doubles memory usage — rarely worth it.

6. Key Recommendations

Key Takeaways

• HNSW dominates IVFFlat for most production use cases — higher recall, lower latency, predictable performance
• Qdrant's native payload indexing makes filtered search 3-5× faster than Pgvector's post-filtering approach
• Pinecone is the right choice when you're optimizing for operational simplicity over raw performance
• Pgvector is viable for datasets under 2M vectors with moderate QPS requirements
• The "right" answer changes as your scale changes — design your data layer to be swappable

Next up: building the hybrid BM25 + dense retrieval pipeline for the RAG system on top of Qdrant.