Overview
| Property | Detail |
|---|
| Pattern | Reranking — The Quality Filter |
| Level | 4 (Better post-retrieval filtering) |
| Key Innovation | Second model re-scores retrieved docs |
| Methods | Cross-encoder (fast, free) + LLM reranking (slow, smarter) |
| Notebook | 09_Reranking.ipynb |
Architecture
flowchart TD
A[User Query] --> B["Stage 1: Embedding Similarity (Qdrant)"]
B --> C["Retrieve TOP-10 by cosine similarity"]
C --> D{"Stage 2: Re-score"}
D -->|Option A| E["Cross-Encoder (ms-marco-MiniLM)\nFast, free, local"]
D -->|Option B| F["LLM Reranking (Claude)\nSlower but understands intent"]
E --> G[Return TOP-3 after re-scoring]
F --> G
Test Results: "What is the capital of France?"
| Rank | Baseline (embedding similarity) | After LLM Reranking |
|---|
| #1 | "The capital of France is huge" (0.87) | "The capital of France is great" (8.0/10) |
| #2 | "The capital of France is great" (0.84) | "The capital of France is beautiful" (7.4/10) |
| #3 | "The capital of France is beautiful" (0.75) | "The capital of France is huge" (6.9/10) |
Key Insight: The LLM scored ALL of these low (4-8/10) because none actually answer "What IS the capital?" — they just mention it. The Paris/Eiffel Tower docs would score much higher with LLM reranking because the LLM understands intent, not just similarity.
Two Reranking Methods
| Aspect | Cross-Encoder | LLM Reranking |
|---|
| Model | ms-marco-MiniLM-L-6-v2 | Claude Sonnet (Bedrock) |
| Speed | Fast (~50ms for 10 docs) | Slow (~2s for 10 docs) |
| Cost | Free (local) | Paid (API call per doc) |
| Understanding | Surface-level relevance | Deep intent matching |
| Best for | High-volume, low-latency | High-stakes, accuracy-critical |
When to Use
| Scenario | Use Reranking? |
|---|
| Top-K retrieval returns noise | Yes |
| Need precision over recall | Yes |
| Can tolerate +100-200ms latency | Yes (cross-encoder) |
| Need intent-aware scoring | Yes (LLM method) |
| Ultra-low latency required | No |
| Already getting good results from embedding search | No — diminishing returns |
Tech Stack
| Component | Technology |
|---|
| Embeddings | AWS Bedrock — Amazon Titan Embed Text v2 (1024 dims) |
| LLM | AWS Bedrock — Claude Sonnet |
| Cross-Encoder | sentence-transformers (ms-marco-MiniLM-L-6-v2) |
| Vector DB | Qdrant Cloud |
| Framework | LangChain |
Source