Overview
HyDE (Hypothetical Document Embeddings) flips traditional RAG: instead of embedding the query, generate a hypothetical ideal answer, embed that, and use it to find similar documents. This bridges the semantic gap between short queries and long documents.
Pattern #5 in the 37 RAG Patterns Collection
The Problem
Query: "How does attention mechanism work?"
→ Embedding matches poorly with detailed technical docs
HyDE Solution:
Query → Generate hypothetical answer → Embed answer → Find similar docs
Architecture
graph TB
A[User Query] --> B[LLM: Generate Hypothetical Answer]
B --> C[Embed Hypothetical Answer]
C --> D[Vector Search]
D --> E[Retrieve Similar Docs]
E --> F[LLM: Generate Final Answer]
F --> G[Response]
H[Vector DB] --> D
style A fill:#f9f,stroke:#333
style B fill:#ff9,stroke:#333
style F fill:#9f9,stroke:#333Why It Works
Query vs Document Mismatch
- ▸Query: "What is RAG?"
- ▸Document: "Retrieval-Augmented Generation (RAG) combines retrieval systems with large language models to provide..."
Queries are short, documents are detailed. Embeddings don't align well.
HyDE Solution
- ▸Generate hypothetical answer (document-like)
- ▸Embed the hypothetical answer
- ▸Find documents semantically similar to the answer
- ▸Use real documents to generate final response
Implementation
Step 1: Generate Hypothetical Document
python
def generate_hypothetical_answer(query):
"""Generate a hypothetical ideal answer"""
prompt = f"""Write a detailed, technical answer to this question:
{query}
Be specific and comprehensive."""
response = bedrock.invoke_model(
modelId="meta.llama3-70b-instruct-v1:0",
body=json.dumps({
"prompt": prompt,
"max_gen_len": 512,
"temperature": 0.7
})
)
return parse_response(response)Step 2: Embed and Search
python
def hyde_retrieval(query, vector_store):
# Generate hypothetical answer
hypothetical = generate_hypothetical_answer(query)
# Embed the hypothetical answer
embedding = get_embedding(hypothetical)
# Search with hypothetical embedding
results = vector_store.search(
embedding=embedding,
top_k=5
)
return resultsStep 3: Generate Final Answer
python
def hyde_rag(query):
# Retrieve using HyDE
docs = hyde_retrieval(query, vector_store)
# Create context from real documents
context = "\n\n".join([doc['content'] for doc in docs])
# Generate final answer with real context
response = bedrock.invoke_model(
modelId="meta.llama3-70b-instruct-v1:0",
body=json.dumps({
"prompt": f"""Context: {context}
Question: {query}
Answer based on the context provided."""
})
)
return responseComplete Notebook
🚀 Run the full implementation:
Includes:
- ▸Complete HyDE pipeline
- ▸Prompt engineering for hypothetical generation
- ▸Comparison with standard RAG
- ▸Performance benchmarks
- ▸Cost analysis
Performance
| Metric | HyDE | Standard RAG |
|---|---|---|
| Accuracy | 85% | 72% |
| Latency | ~600ms | ~300ms |
| Cost/query | $0.15 | $0.08 |
| Best for | Complex queries | Simple Q&A |
When to Use
Best For
- ▸Complex, technical queries requiring domain expertise
- ▸Ambiguous questions needing interpretation
- ▸Research applications where precision matters
- ▸Domain-specific search (medical, legal, scientific)
Avoid When
- ▸Simple factual queries (unnecessary overhead)
- ▸Real-time requirements (adds LLM generation latency)
- ▸Budget constraints (2x LLM calls per query)
Optimization Tips
1. Cache Hypothetical Answers
python
import hashlib
cache = {}
def hyde_with_cache(query):
query_hash = hashlib.md5(query.encode()).hexdigest()
if query_hash not in cache:
cache[query_hash] = generate_hypothetical_answer(query)
return cache[query_hash]2. Multi-Hypothetical HyDE
python
def multi_hyde(query, n=3):
"""Generate multiple hypothetical answers for diversity"""
hypotheticals = [
generate_hypothetical_answer(query)
for _ in range(n)
]
# Search with all hypotheticals
all_results = []
for hyp in hypotheticals:
results = vector_store.search(get_embedding(hyp))
all_results.extend(results)
# Deduplicate and rank
return deduplicate_and_rank(all_results)3. Hybrid: HyDE + Direct Query
python
def hybrid_hyde(query):
# Get results from both methods
hyde_results = hyde_retrieval(query)
direct_results = vector_store.search(get_embedding(query))
# Combine with fusion
combined = reciprocal_rank_fusion([hyde_results, direct_results])
return combinedComparison
| Approach | Pros | Cons |
|---|---|---|
| Standard RAG | Fast, cheap | Poor for complex queries |
| HyDE | Better accuracy | Slower, more expensive |
| Hybrid HyDE | Balanced | More complex |
Related Papers
- ▸HyDE: Precise Zero-Shot Dense Retrieval (2022)
- ▸Query2Doc: Similar approach (2023)
Next Steps
- ▸Try it: Run the HyDE notebook
- ▸Combine: Use with Reranking (#4) for best results
- ▸Optimize: Add Multi-Query (#9) for diversity
- ▸Compare: Benchmark against your baseline
Part of: 37 RAG Patterns Collection
See also: Visual Architecture Guide