Measuring Search Quality: NDCG, Recall, and A/B Testing
You can't improve what you can't measure. Search quality is subjective ("are these results good?") but it can be made quantitative. The two metrics that matter most in practice are NDCG@k and Recall@k.
NDCG@k: the gold standard
Normalized Discounted Cumulative Gain measures both whether the right results are returned AND whether they're ranked in the right order.
The key insight: a relevant result in position 1 is worth more than the same result in position 10. Users click the top results; they rarely scroll to position 8.
DCG@k = Σ(i=1 to k) relevance_i / log2(i + 1)
NDCG@k = DCG@k / IDCG@k
Where IDCG@k is the ideal DCG (perfect ranking). NDCG is normalized to [0, 1] so results are comparable across queries.
Example: query "vector search tutorial"
| Position | Document | Relevance | Discount | Contribution |
|---|---|---|---|---|
| 1 | "Intro to Vector Search" | 3 (highly relevant) | 1.0 | 3.0 |
| 2 | "HNSW Explained" | 2 (relevant) | 0.63 | 1.26 |
| 3 | "SQL Basics" | 0 (irrelevant) | 0.50 | 0.0 |
| 4 | "Embeddings Guide" | 2 (relevant) | 0.43 | 0.86 |
NDCG@4 = (3.0 + 1.26 + 0 + 0.86) / IDCG@4
Typical production NDCG@10 benchmarks:
- ▸Keyword only: 0.72
- ▸Dense only: 0.85
- ▸Hybrid: 0.91
Recall@k: are the right results in the top-k?
Recall@k measures what fraction of relevant documents appear in the top-k results:
Recall@10 = (relevant docs in top-10) / (total relevant docs in corpus)
Use Recall@k to measure your HNSW configuration: compare approximate kNN (your production index) vs. exact kNN (brute force) on a sample of queries:
Recall@10 = |results_approx ∩ results_exact| / k
ef_search=50: Recall@10 ≈ 0.92
ef_search=100: Recall@10 ≈ 0.96
ef_search=200: Recall@10 ≈ 0.98
A Recall@10 of 0.96 means that for every 10 results your system returns, 9.6 on average are the same ones exact kNN would return. That 0.4 "miss" is the cost of approximation.
Building a ground-truth dataset
You need labeled query-result pairs to compute NDCG. Three ways to collect them:
1. Human annotation (highest quality) Present evaluators with queries and candidate results. Ask them to rate relevance on a 0–3 scale (0=irrelevant, 1=somewhat, 2=relevant, 3=highly relevant). 200–500 queries is sufficient for reliable NDCG estimates.
2. Click data (easy at scale) If you have existing search traffic, treat clicks as implicit relevance signals. Position-correct for click bias (results at position 1 get clicked more regardless of relevance):
relevance = click / (1 / position) ← position-corrected click rate
3. LLM annotation (fast and cheap) Send (query, document) pairs to an LLM and ask for relevance scores. Correlates well with human annotation for most domains. Validate against a small human-annotated sample before trusting it.
[Key Insight] Start with 200 manually annotated queries. That's enough to detect a 3–5% NDCG difference with statistical significance. Anything smaller than that difference isn't worth optimizing for.
Benchmarking your pipeline
Run evaluation before and after any change:
For each query in evaluation set:
1. Run search with current system → results_current
2. Run search with new system → results_new
3. Compute NDCG@10(results_current, ground_truth)
4. Compute NDCG@10(results_new, ground_truth)
Summary: mean NDCG improvement, breakdown by query type
Always break down NDCG by query type (exact-term vs. semantic vs. navigational). Aggregate NDCG can improve while one query type gets worse — which your users will notice even if the number looks good.
Latency benchmarking
Measure P50, P95, and P99 latency separately. P95 and P99 matter more than mean — they're what your slowest users experience:
Collect 1,000+ query latencies
P50 (median): comfortable estimate of typical experience
P95: 1 in 20 queries hits this or worse
P99: 1 in 100 queries — your tail latency
Alert on P95 > 200ms for interactive search
Alert on P99 > 500ms
Index health degrades over time as new documents are added and segments fragment. Schedule weekly NDCG + P95 measurement. If P95 spikes, check segment count — you may need to trigger re-optimization.
A/B testing weight combinations
To tune hybrid search weights, run controlled experiments:
User group A: hybrid weights (0.3, 0.3, 0.4) ← control
User group B: hybrid weights (0.4, 0.3, 0.3) ← treatment
Measure: click-through rate, session length, zero-result rate
Run for minimum 2 weeks (captures day-of-week variation)
Require p < 0.05 for significance
For offline evaluation (using your labeled dataset), a grid search is faster:
For each weight combination in grid:
(0.1, 0.1, 0.8), (0.2, 0.2, 0.6), (0.3, 0.3, 0.4), ...
Compute NDCG@10 on evaluation set
Pick the combination with highest average NDCG
Offline grid search identifies candidates; online A/B confirms the winner holds on real traffic.
What to monitor in production
Once live, track:
- ▸NDCG@10 (weekly offline eval): did quality drift?
- ▸Zero-result rate: are users hitting dead ends?
- ▸P95 latency: is the index degrading?
- ▸Click depth: how far are users scrolling before clicking?
- ▸Reformulation rate: how often do users immediately modify their query?
Reformulation rate is a strong signal: if a user searches "payment error", clicks nothing, then immediately searches "payment processing error", the first query failed.