Overview
Contextual Compression uses an LLM to filter and compress retrieved documents, removing irrelevant information while keeping only what's needed to answer the query. This reduces token usage and improves response quality.
Pattern #6 in the 37 RAG Patterns Collection
flowchart TD
Q[Query] --> R["Retrieve top-K chunks standard kNN"]
R --> CC{"For each chunk: LLM extract only sentences relevant to the query"}
CC --> EMPTY{Is extracted text empty?}
EMPTY -- yes --> DROP[Drop chunk entirely]
EMPTY -- no --> KEEP[Keep compressed excerpt]
KEEP --> PACK[Pack compressed chunks into prompt]
PACK --> LLM[Generate — smaller prompt, same relevance]The Problem
Standard RAG retrieves full documents:
Query: "What is the capital of France?"
Retrieved Document (500 tokens):
"France, officially the French Republic, is a country located in Western Europe.
It has a population of 67 million people and covers an area of 643,801 km².
The country has a rich history dating back thousands of years...
[400 more tokens of irrelevant information]
...The capital city is Paris, located in the north-central part of the country."
Problem: Only 5 tokens were relevant, but sent 500 tokens!
Solution
Compress before generation:
Query: "What is the capital of France?"
↓
Retrieve: [500-token document]
↓
Compress: Extract only relevant portions
↓
Result: "The capital city is Paris."
↓
Generate answer with compressed context
Implementation
python
def compress_context(query, documents, llm):
"""Compress retrieved documents to relevant portions only"""
compressed = []
for doc in documents:
prompt = f"""Query: {query}
Document: {doc['content']}
Extract ONLY the sentences directly relevant to answering the query.
Remove all unrelated information.
Return only the essential facts."""
compressed_text = llm.invoke(prompt)
if len(compressed_text.strip()) > 0:
compressed.append({
'content': compressed_text,
'source': doc['source'],
'compression_ratio': len(compressed_text) / len(doc['content'])
})
return compressed
def contextual_compression_rag(query):
# Step 1: Retrieve
documents = retriever.search(query, top_k=10)
# Step 2: Compress
compressed = compress_context(query, documents, llm)
# Step 3: Generate with compressed context
context = "\n\n".join([c['content'] for c in compressed])
response = llm.generate(
f"Context: {context}\n\nQuestion: {query}"
)
return response, {
'original_tokens': sum(len(d['content']) for d in documents),
'compressed_tokens': len(context),
'compression_ratio': len(context) / sum(len(d['content']) for d in documents),
'documents_used': len(compressed)
}Complete Notebook
🚀 Run the full implementation:
📓 Contextual Compression AWS Notebook
Performance
| Metric | With Compression | Without |
|---|---|---|
| Tokens/query | 300 | 1,500 |
| Cost | $0.10 | $0.18 |
| Accuracy | 88% | 85% |
| Latency | ~500ms | ~800ms |
Compression saves 80% tokens while improving accuracy!
When to Use
Best For
- ▸High token costs - Reduce generation costs
- ▸Long documents - Remove fluff, keep facts
- ▸Precise queries - Clear information need
- ▸Multi-document retrieval - Compress from many sources
Combine With
- ▸Reranking (#4) - Compress after reranking
- ▸Semantic Chunking (#7) - Better initial chunks
- ▸Caching (#33) - Cache compressed results
Compression Strategies
1. Extractive (Fast)
python
# Extract only relevant sentences
compressed = extract_relevant_sentences(doc, query)2. Abstractive (Better Quality)
python
# Rewrite in compressed form
compressed = llm.summarize(doc, focus=query)3. Hybrid (Best)
python
# Extract, then compress further
extracted = extract_relevant_sentences(doc, query)
compressed = llm.compress(extracted)Related Papers
- ▸Compressing Context: LLMLingua (2023)
- ▸Selective Context: Information Bottleneck (2021)
Next Steps
- ▸Try it: Run the notebook
- ▸Optimize: Combine with Reranking (#4)
- ▸Scale: Use Cached RAG (#33) for repeated queries
Part of: 37 RAG Patterns Collection