Hybrid Search: Combining Dense, Sparse, and Keyword
No single search method wins on all query types. Dense vectors excel at semantic meaning. Sparse encoding handles exact technical terms. Keyword (BM25) search is fastest and most predictable. Hybrid search combines all three — and the quality improvement is consistent and significant.
| Engine | NDCG@10 overall | Exact terms | Synonyms | Concepts |
|---|---|---|---|---|
| Keyword (BM25) | 0.72 | 0.95 | 0.35 | 0.32 |
| Sparse (TF-IDF) | 0.74 | 0.89 | 0.52 | 0.47 |
| Dense (embeddings) | 0.85 | 0.68 | 0.91 | 0.93 |
| Hybrid (all three) | 0.91 | 0.96 | 0.89 | 0.91 |
The 7% lift from 0.85 to 0.91 is the result of each engine catching what the others miss.
The three engines
Keyword (BM25): Exact term matching with inverse document frequency weighting. Fastest engine (2ms P95). Dominates on precise, controlled vocabulary queries. Fails completely on paraphrasing.
Sparse (TF-IDF + expansion): Term-weighted sparse vectors. Handles domain vocabulary plus light synonym expansion. 5ms P95. Bridges keyword and semantic.
Dense (embeddings + HNSW): Full semantic understanding. Best for natural language queries, paraphrasing, multi-language. 45ms P95. Fails on exact codes and IDs.
The combination problem
Scores from different engines are not comparable. BM25 scores might range 0–15. Cosine similarity ranges -1 to 1. Before combining, normalize each engine's results to [0, 1]:
normalized_score = (score - min_score) / (max_score - min_score)
Applied per-query, per-engine, over the result set being merged.
The combination formula
After normalization, combine with weighted sum:
combined_score = w_keyword × score_kw + w_sparse × score_sp + w_dense × score_dn
Default weights: (0.3, 0.3, 0.4) ← dense gets slightly more weight
Reciprocal Rank Fusion (RRF)
RRF is the industry-standard alternative to weighted sum, and it's the default fusion method used when multiple queries run in parallel in most production search systems.
Instead of combining raw scores, RRF combines rankings. Each engine votes for documents based on their rank position:
RRF_score(doc) = Σ 1 / (k + rank_in_engine_i)
where k = 60 (smoothing constant, standard default)
For each engine, a document ranked 1st contributes 1/(60+1) = 0.0164, ranked 10th contributes 1/(60+10) = 0.0143, ranked 100th contributes 1/(60+100) = 0.0063.
Why RRF is robust: it doesn't require score normalization (rank-based, not score-based), it's insensitive to score scale differences between engines, and it naturally down-weights documents that appear in only one engine's results.
Query: "machine learning optimization"
Keyword results: rank 1 = Doc A, rank 2 = Doc B, rank 5 = Doc C
Dense results: rank 1 = Doc C, rank 2 = Doc A, rank 8 = Doc D
RRF scores:
Doc A: 1/61 + 1/62 = 0.0325 (top-2 in both → highest)
Doc C: 1/65 + 1/61 = 0.0318 (top-5 keyword + top-1 dense)
Doc B: 1/62 = 0.0161 (only in keyword)
Doc D: 1/68 = 0.0147 (only in dense, ranked 8th)
Documents that appear in both engines and rank highly in both float to the top. Documents only one engine finds rank lower.
[Key Insight] Use RRF when you don't have labeled data to tune weights — it works well out of the box with no calibration. Use weighted sum when you have a labeled evaluation set and want to optimize specifically for your query distribution. In practice, well-tuned weighted sum outperforms RRF by 2–5% NDCG on domain-specific corpora.
Oversampling for better fusion
When merging results from multiple engines, each engine may return different documents in its top-k. A document that ranks 15th in dense search might rank 3rd in keyword search — but if you only asked each engine for top-10, you'd miss it.
Oversampling addresses this: request more candidates from each engine than you need in the final result, fuse them, then return the true top-k:
Final result needed: top-10
Oversample factor: 3–5×
Request from each engine: top-30 to top-50
Fuse all candidates (union across engines)
Return top-10 after fusion
A 3–5× oversampling factor captures nearly all documents that would rank in the top-10 after fusion. The cost is marginal — you're running 3–5× more scoring comparisons against the inverted indexes, but the HNSW traversal cost (the expensive part) doesn't change.
Combination methods
Six approaches are commonly used:
| Method | Formula | When to use |
|---|---|---|
| Weighted sum | Σ(wi × si) | Default — tunable per domain |
| Arithmetic mean | Σsi / n | Equal trust in all engines |
| Harmonic mean | n / Σ(1/si) | Penalizes when engines disagree |
| Geometric mean | (Π si)^(1/n) | Requires consensus across engines |
| Max | max(si) | Any one engine is sufficient |
| Min | min(si) | All engines must agree (conservative) |
Weighted sum is the right default. It's interpretable, tunable, and has no degenerate edge cases.
Weight tuning by use case
The default (0.3, 0.3, 0.4) is a starting point. Tune for your query distribution:
| Use case | Keyword | Sparse | Dense | Reasoning |
|---|---|---|---|---|
| E-commerce catalog | 0.4 | 0.3 | 0.3 | SKU/product name precision critical |
| Technical documentation | 0.5 | 0.3 | 0.2 | Function names, error codes dominate |
| Customer Q&A | 0.2 | 0.3 | 0.5 | User phrasing varies widely |
| Knowledge base | 0.3 | 0.3 | 0.4 | Balanced |
[Key Insight] Tune weights empirically: collect 200–500 labeled query-result pairs (user clicks, explicit relevance judgments), compute NDCG@10 across a grid of weight combinations, pick the highest. Re-tune every quarter or after a major corpus change.
Architecture
User query
│
├─► Keyword index (inverted index) ─────────────────────┐
├─► Sparse encoder + inverted index ────────────────────┤ → Normalize → Combine → Rank
└─► Embedding model → HNSW index ──────────────────────┘
Run all three concurrently: wall-clock ~35ms (not 52ms sum)
The three retrieval paths run in parallel. Wall-clock time is dominated by the slowest engine (dense at ~45ms), not the sum of all three.
Why it works
Each engine catches what the others miss:
- ▸Query: "fast concurrent code" → dense finds "parallel programming" (synonym); keyword finds "concurrent.futures" (exact term)
- ▸Query: "prod error NullPointerException" → keyword/sparse find the exact exception; dense finds related debugging articles
- ▸Query: "how to reduce memory usage" → dense understands "reduce memory" = "optimize memory"; keyword/sparse find explicit "memory" matches
No single engine covers all three. Together they do.
Performance summary
| Engine | Latency P95 | Memory (1M docs) | NDCG@10 |
|---|---|---|---|
| Keyword | 2ms | 200 MB | 0.72 |
| Sparse | 5ms | 300 MB | 0.74 |
| Dense | 45ms | 3 GB | 0.85 |
| Hybrid | 35ms | 3.6 GB | 0.91 |
Hybrid is actually faster than dense alone at P95 (35ms vs 45ms) because concurrent execution and early termination reduce tail latency.