Ensemble Retrieval - Combining Multiple Retrieval Strategies

Overview

Ensemble Retrieval runs several independent retrieval strategies in parallel — dense vector search, BM25 sparse retrieval, metadata-filtered lookup — and merges their ranked result lists using Reciprocal Rank Fusion (RRF). No single retriever wins on all query types; ensemble methods consistently outperform any individual strategy by combining their complementary strengths.

Pattern #19 in the 37 RAG Patterns Collection

flowchart TD
  Q[Query] --> S1[Strategy 1: Simple RAG embed → search]
  Q --> S2["Strategy 2: HyDE generate hypothesis → embed → search"]
  Q --> S3["Strategy 3: Fusion N phrasings → individual searches → RRF"]
  S1 --> RRF["Reciprocal Rank Fusion k=60 across all three result sets"]
  S2 --> RRF
  S3 --> RRF
  RRF --> TOP[Top-K fused results]
  TOP --> LLM[Generate — best quality, highest cost]

The Problem

Every retriever has a blind spot:

Query: "PEP 484 type hints introduced in Python 3.5"

Dense vector search (semantic):
  → Finds conceptually related content about Python typing
  → Misses the exact "PEP 484" acronym match            ❌

BM25 sparse search (keyword):
  → Exact match on "PEP 484" and "Python 3.5"           ✅
  → Misses documents that discuss type hints without using
    the PEP number                                        ❌

Metadata filter (date/version):
  → Finds all Python 3.5 release notes                  ✅
  → Returns many off-topic documents                     ❌

The best answer often appears in different rank positions across retrievers. Merging the lists surfaces it reliably.

Solution: Reciprocal Rank Fusion

RRF assigns each document a score based on its rank in each retrieval list, then sums across retrievers:

score(document) = Σ  1 / (k + rank_i(document))
                 i

k = 60  (constant that dampens the influence of very high ranks)

A document ranked #1 in dense and #5 in BM25 scores better than one ranked #3 in only one list — rank consistency across retrievers is the signal.

Implementation

python

import asyncio
from collections import defaultdict

async def run_retrievers_concurrently(query: str) -> dict[str, list[dict]]:
    """Run all retrievers in parallel and collect ranked result lists."""
    dense_task  = asyncio.create_task(vector_store.search_async(query, top_k=20))
    bm25_task   = asyncio.create_task(bm25_index.search_async(query, top_k=20))
    filter_task = asyncio.create_task(metadata_store.search_async(query, top_k=20))

    dense, bm25, filtered = await asyncio.gather(dense_task, bm25_task, filter_task)

    return {"dense": dense, "bm25": bm25, "metadata": filtered}


def reciprocal_rank_fusion(
    ranked_lists: dict[str, list[dict]],
    k: int = 60,
    top_k: int = 10,
) -> list[dict]:
    """
    Merge ranked lists from multiple retrievers using RRF.
    ranked_lists: {"retriever_name": [{"id": str, "content": str, ...}, ...]}
    Returns deduplicated list sorted by fused score descending.
    """
    scores: dict[str, float] = defaultdict(float)
    doc_map: dict[str, dict] = {}

    for retriever_name, results in ranked_lists.items():
        for rank, doc in enumerate(results, start=1):
            doc_id = doc["id"]
            scores[doc_id] += 1.0 / (k + rank)
            doc_map[doc_id] = doc  # keep last-seen copy (all identical by id)

    # Sort by fused score descending
    sorted_ids = sorted(scores, key=lambda d: scores[d], reverse=True)

    return [
        {**doc_map[doc_id], "rrf_score": scores[doc_id]}
        for doc_id in sorted_ids[:top_k]
    ]


async def ensemble_rag(query: str) -> tuple[str, dict]:
    # Step 1: Retrieve from all sources in parallel
    ranked_lists = await run_retrievers_concurrently(query)

    # Step 2: Fuse with RRF
    fused_docs = reciprocal_rank_fusion(ranked_lists, k=60, top_k=10)

    # Step 3: Generate
    context = "\n\n".join(doc["content"] for doc in fused_docs)
    response = llm.generate(f"Context: {context}\n\nQuestion: {query}")

    return response, {
        "dense_count":    len(ranked_lists["dense"]),
        "bm25_count":     len(ranked_lists["bm25"]),
        "metadata_count": len(ranked_lists["metadata"]),
        "fused_count":    len(fused_docs),
        "top_rrf_score":  fused_docs[0]["rrf_score"] if fused_docs else 0,
    }

Weighted RRF Variant

When one retriever is known to be more reliable for your domain, apply a weight multiplier:

python

def weighted_rrf(
    ranked_lists: dict[str, list[dict]],
    weights: dict[str, float],
    k: int = 60,
) -> list[dict]:
    """
    weights example: {"dense": 1.5, "bm25": 1.0, "metadata": 0.8}
    """
    scores: dict[str, float] = defaultdict(float)
    doc_map: dict[str, dict] = {}

    for name, results in ranked_lists.items():
        w = weights.get(name, 1.0)
        for rank, doc in enumerate(results, start=1):
            doc_id = doc["id"]
            scores[doc_id] += w * (1.0 / (k + rank))
            doc_map[doc_id] = doc

    sorted_ids = sorted(scores, key=lambda d: scores[d], reverse=True)
    return [{**doc_map[i], "rrf_score": scores[i]} for i in sorted_ids]

Complete Notebook

Run the full implementation:

Ensemble Retrieval AWS Notebook

Includes:

▸Dense + BM25 + metadata retriever setup on AWS
▸Standard and weighted RRF implementations
▸Overlap analysis across retrievers
▸Ablation study comparing individual vs ensemble recall
▸Integration with Amazon OpenSearch for BM25

Performance

Metric	Dense Only	BM25 Only	Ensemble (RRF)
Recall@10	71%	68%	84%
Precision@5	65%	60%	74%
MRR@10	0.62	0.58	0.76
Latency	~80ms	~40ms	~85ms (parallel)

Ensemble recall jumps to 84% — 13 points above the best individual retriever — with minimal extra latency when run in parallel.

When to Use

Best For

▸Broad knowledge bases with heterogeneous content — some documents use precise terminology, others use natural language
▸Mixed content types — structured product records alongside unstructured support articles benefit from both keyword and semantic matching
▸High-recall requirements — legal or compliance search where missing a relevant document has real consequences
▸Unknown query distribution — when you cannot predict whether users will issue keyword or conversational queries

Combine With

▸Reranking (#4) — use RRF to maximise recall over a large merged candidate set, then run a cross-encoder to precision-score the top-20 before passing to the LLM
▸Adaptive RAG (#8) — route simple keyword queries to BM25 alone, semantic queries to dense alone, and ambiguous queries to the full ensemble to avoid unnecessary latency

Next Steps

▸Try it: Run the notebook
▸Add reranking: Pipe the RRF output through Cross-Encoder Reranking (#4) for precision at the top of the list
▸Tune weights: Measure per-retriever recall on a dev set and set weights accordingly in Weighted RRF
▸Reduce latency: Ensure all retrievers run concurrently with asyncio.gather — sequential ensemble defeats its purpose

Part of: 37 RAG Patterns Collection See also: Complete Pattern Index