33 RAG Patterns — Complete Tier Guide

RAG is not a single technique. It's a family of 33 distinct patterns spanning nine tiers, from a simple chunk-and-retrieve baseline to multi-tenant federated systems. This guide maps all of them — what each does, how it differs from the baseline, and links to working notebooks.

The baseline: Simple RAG

Before the tiers, understand what everything improves on:

flowchart LR
  A[PDF] --> B[Split into chunks]
  B --> C[Embed chunks]
  C --> D[(Vector Index)]
  E[Query] --> F[Embed query]
  F --> G[kNN Search]
  D --> G
  G --> H[Top-K chunks]
  H --> I[LLM Generate]
  I --> J[Answer]

Weaknesses of simple RAG: fixed-size chunks lose context at boundaries · single embedding angle misses paraphrased queries · no quality check on retrieved content · stateless · not production-ready.

The nine tiers each address a specific weakness.

Tier 1 — Chunking & Indexing (NB 01–06)

How you split and index documents determines your retrieval ceiling. No retrieval strategy recovers from bad chunks.

#	Pattern	Key idea	Notebook
01	Simple RAG	Fixed-size chunks + cosine kNN — the baseline	01_Simple_RAG.ipynb
02	Semantic Chunking	Break at topic-shift, not character count	02_Semantic_Chunking.ipynb
03	Hierarchical RAG	Small child chunks for retrieval, large parent for context	03_Hierarchical_RAG.ipynb
04	Parent-Child RAG	4-level tree: doc → section → paragraph → sentence	04_Parent_Child_RAG.ipynb
05	Sentence Window RAG	Index sentences, expand ±W neighbours at query time	05_Sentence_Window_RAG.ipynb
06	Contextual Retrieval	LLM prepends a location summary to each chunk before embedding	06_Contextual_Retrieval.ipynb

Semantic Chunking — flow

flowchart TD
  A[PDF text] --> B[Split into sentences]
  B --> C[Embed each sentence]
  C --> D[Compute adjacent cosine similarity]
  D --> E{Similarity drop below threshold?}
  E -- Yes: topic shift --> F[Start new chunk here]
  E -- No: same topic --> G[Continue current chunk]
  F --> H[Embed semantic chunks]
  G --> H
  H --> I[(Vector Index)]

Hierarchical RAG — flow

flowchart TD
  A[PDF] --> B["Parent chunks ~1000 chars"]
  B --> C["Child chunks ~200 chars per parent"]
  C --> D[Embed children only]
  D --> E[(Index children with parent_id payload)]
  F[Query] --> G[Embed query]
  G --> H[Search children top-K]
  E --> H
  H --> I[Resolve parent_id for each hit]
  I --> J[Fetch full parent text]
  J --> K[Deduplicate parents]
  K --> L[LLM Generate]

Contextual Retrieval — flow

flowchart TD
  A[PDF] --> B["Fixed chunks 500 chars"]
  B --> C{For each chunk at index time}
  C --> D["LLM: write 1-2 sentences describing where this chunk sits in the document"]
  D --> E[Prepend context to chunk text]
  E --> F[Embed enriched text]
  F --> G[(Index)]
  H[Query] --> I[Embed] --> J[kNN Search]
  G --> J
  J --> K[LLM Generate]

[Key Insight] Contextual Retrieval (NB 06) reduces retrieval failures by 49–67% per Anthropic research. The index-time LLM call costs roughly $0.01 per 1,000 chunks — usually worth it for knowledge bases where recall matters.

Tier 2 — Retrieval Quality (NB 07–12)

Better search ranking within the same index.

#	Pattern	Key idea	Notebook
07	Hybrid Search	BM25 + vector + RRF fusion	07_Hybrid_Search.ipynb
08	HyDE	Embed a generated hypothetical answer, not the query	08_HyDE.ipynb
09	Reranking	LLM re-scores top-20 candidates, keeps top-5	09_Reranking.ipynb
10	Contextual Compression	Strip off-topic sentences from retrieved chunks	10_Contextual_Compression.ipynb
11	Metadata Filtering	Pre-filter by section/topic/page before vector search	11_Metadata_Filtering.ipynb
12	Multi-Document RAG	Global / scoped / parallel retrieval across multiple docs	12_Multi_Document_RAG.ipynb

Hybrid Search — flow

flowchart TD
  Q[Query] --> V[Embed query]
  Q --> K[Tokenise query]
  V --> VS["Vector search top-20"]
  K --> BS["BM25 keyword search top-20"]
  VS --> R["Reciprocal Rank Fusion RRF k=60"]
  BS --> R
  R --> T[Top-5 fused results]
  T --> G[LLM Generate]

HyDE — flow

flowchart TD
  Q[Sparse query] --> H["LLM: write a 150-word hypothetical answer in textbook style"]
  H --> HE[Embed hypothetical document]
  HE --> S[kNN search with hypothetical embedding]
  S --> RC[Retrieve real document chunks]
  RC --> G[LLM Generate with real evidence]

Reranking — 2-stage flow

flowchart TD
  Q[Query] --> E[Embed]
  E --> S["Recall stage: kNN top-20"]
  S --> R{LLM scores each chunk 0-10}
  R --> SO[Sort by LLM score]
  SO --> T[Top-5 reranked chunks]
  T --> G[LLM Generate]

Tier 3 — Query Handling (NB 13–17)

Transform the question before retrieval to cover more ground.

#	Pattern	Key idea	Notebook
13	Query Decomposition	Break complex question into N sub-questions	13_Query_Decomposition.ipynb
14	Step-Back Prompting	Retrieve background principle + specific answer	14_Step_Back_Prompting.ipynb
15	RAG Fusion	N query phrasings → individual searches → RRF merge	15_Fusion_Retrieval.ipynb
16	Chain-of-Thought RAG	Sequential retrieval: each step informed by prior findings	16_Chain_of_Thought_RAG.ipynb
17	ReAct RAG	LLM dynamically chooses: search / keyword lookup / finish	17_ReAct_RAG.ipynb

Query Decomposition — flow

flowchart TD
  Q[Complex multi-part question] --> D["LLM: decompose into N sub-questions max 4"]
  D --> Q1[Sub-question 1] & Q2[Sub-question 2] & Q3[Sub-question N]
  Q1 --> S1[Search top-4]
  Q2 --> S2[Search top-4]
  Q3 --> S3[Search top-4]
  S1 & S2 & S3 --> M[Merge + deduplicate by chunk ID]
  M --> G[LLM Synthesise final answer]

Chain-of-Thought RAG — flow

flowchart TD
  Q[Complex question] --> P["LLM: plan ordered reasoning steps JSON max 5"]
  P --> L{Sequential loop}
  L --> QS["Query = step description + last 2 prior findings"]
  QS --> R[Retrieve top-4]
  R --> F["LLM reasons → intermediate finding"]
  F --> L
  L --> |all steps done| SY[Synthesise all findings → final answer]

ReAct RAG — flow

flowchart TD
  Q[Question] --> T[Thought: what do I know / need?]
  T --> A{Choose action}
  A -- search query --> VS[Vector search → Observation]
  A -- lookup term --> KW[Keyword search → Observation]
  A -- finish --> ANS[Return answer]
  VS --> T
  KW --> T
  T -.->|MAX 8 iterations| FORCE[Force synthesis]

Tier 4 — Agentic & Self-Improving (NB 18–22)

The system evaluates and corrects its own retrieval and output.

#	Pattern	Key idea	Notebook
18	Corrective RAG	Grade chunks 0-10, route to refine/fallback if low quality	18_Corrective_RAG.ipynb
19	Self RAG	Iteratively refine answer on 4 quality dimensions	19_Self_RAG.ipynb
20	Iterative RAG	Gap analysis → fetch missing evidence → re-generate	20_Iterative_RAG.ipynb
21	Recursive RAG	Low-confidence chunks resolved to parent section at query time	21_Recursive_RAG.ipynb
22	Agentic RAG	LLM autonomously picks retrieval tools via Strands framework	22_Agentic_RAG.ipynb

Corrective RAG (CRAG) — flow

flowchart TD
  Q[Query] --> R["Retrieve top-6"]
  R --> G{Grade each chunk 0-10}
  G --> AVG[Compute average score]
  AVG --> RT{Route based on average}
  RT -- avg ≥ 0.7 --> USE[Use as-is → Generate]
  RT -- "0.4–0.7" --> REF["Refine: keep good chunks, rewrite query for bad slots, re-retrieve"]
  RT -- avg < 0.4 --> FALL["Fallback: broad query rewrite, full re-retrieve"]
  REF --> GEN[LLM Generate]
  FALL --> GEN

Self RAG — iterative refinement flow

flowchart TD
  Q[Query] --> R[Retrieve chunks fixed]
  R --> D[Generate draft answer]
  D --> C{"Critique: faithfulness / completeness / conciseness / groundedness scored 0-1"}
  C -- all dimensions pass threshold --> OUT[Return answer]
  C -- any dimension fails --> IMP["Generate again with targeted improvement hint for weakest dimension"]
  IMP --> C
  C -.->|MAX 3 iterations| OUT

Iterative RAG — gap-filling flow

flowchart TD
  Q[Query] --> R["Initial retrieve top-5"]
  R --> D[Generate draft]
  D --> GAP["Gap analysis: what is still missing? JSON array max 3"]
  GAP --> |no gaps identified| OUT[Return answer]
  GAP --> |gaps found| FE["Retrieve fresh evidence per gap top-3"]
  FE --> M[Merge + deduplicate evidence pool]
  M --> D2[Regenerate refined answer]
  D2 --> GAP
  GAP -.->|MAX 3 iterations| OUT

Tier 5 — Memory & Conversation (NB 23–24)

Stateful RAG that remembers prior turns.

#	Pattern	Key idea	Notebook
23	Memory-Augmented RAG	Sliding-window history + pronoun resolution before embedding	23_Memory_Augmented_RAG.ipynb
24	Multi-Turn Conversational RAG	History as embeddings in a second collection — semantic recall	24_Multi_Turn_Conversational_RAG.ipynb

Multi-Turn Conversational RAG — flow

flowchart TD
  Q[User turn N query] --> E[Embed query]
  E --> DS["Search docs collection top-4"]
  E --> HS["Search history collection top-2 semantic recall"]
  DS --> P["Build prompt: HISTORY block + DOCS block + QUESTION"]
  HS --> P
  P --> G[LLM Generate]
  G --> ANS[Answer to user]
  ANS --> SE["Embed concatenation: Q: ... A: ..."]
  SE --> HC[(Store turn in history collection)]

[Key Insight] NB 24 stores conversation history as embeddings in a second collection. Unlike a sliding window, this enables semantic retrieval of relevant past turns — "the second point you mentioned" retrieves the actual second point even if it was 20 turns ago.

Tier 6 — Ensemble & Adaptive (NB 25–26)

Run multiple strategies and select or combine the best.

#	Pattern	Key idea	Notebook
25	Ensemble RAG	Simple + HyDE + Fusion all run → RRF merge	25_Ensemble_RAG.ipynb
26	Adaptive RAG	LLM classifies query type → routes to best single strategy	26_Adaptive_RAG.ipynb

Adaptive RAG — routing flow

flowchart TD
  Q[Query] --> CL{"LLM classifier max_tokens=10"}
  CL -- factual --> SR[Simple RAG: embed query → search]
  CL -- conceptual --> HY[HyDE: generate hypothesis → embed → search]
  CL -- comparative --> FU[Fusion RAG: N phrasings → RRF]
  CL -- exploratory --> FU
  SR & HY & FU --> G[LLM Generate]

[Key Insight] Ensemble (NB 25) = best quality, 3× cost. Adaptive (NB 26) = near-best quality, 1× cost. The classification call is max_tokens=10 — negligible overhead. Start with Adaptive in production.

Tier 7 — Production (NB 27–30)

Streaming, caching, automated evaluation, and multi-modal.

#	Pattern	Key idea	Notebook
27	Streaming RAG	Tokens surface immediately via response stream	27_Streaming_RAG.ipynb
28	Caching RAG	L1 SHA-256 hash + L2 semantic similarity — skip duplicate queries	28_Caching_RAG.ipynb
29	Evaluation RAG	Auto-generate Q&A → Hit@K / MRR / NDCG + LLM judge	29_Evaluation_RAG.ipynb
30	Complete Pipeline	Multi-modal: text + tables + chart images in one index	30_Complete_Pipeline_RAG.ipynb

Caching RAG — two-level flow

flowchart TD
  Q[Query] --> L1{"L1: SHA-256 exact hash match"}
  L1 -- HIT: 0 API calls --> ANS[Return cached answer instantly]
  L1 -- MISS --> E[Embed query: 1 embedding call]
  E --> L2{"L2: cosine similarity ≥ 0.75 vs stored embeddings"}
  L2 -- HIT: 1 embedding call total --> ANS
  L2 -- MISS --> FULL[Full retrieve + generate pipeline]
  FULL --> STORE[Store in both L1 and L2 cache]
  STORE --> ANS

Evaluation RAG — closed-loop measurement flow

flowchart TD
  A[(Index)] --> S[Sample N chunks evenly across document]
  S --> QG["LLM: generate 1 factual question per chunk"]
  QG --> EV{For each eval question}
  EV --> RET[Retrieve top-K]
  RET --> METRICS["Hit@K · MRR · NDCG@K"]
  EV --> GEN[Full RAG answer]
  GEN --> JUDGE["LLM judge: faithfulness 1-5 + answer relevance 1-5"]
  METRICS & JUDGE --> REPORT[Evaluation report with latency breakdown]

Tier 8 — Incremental Indexing (NB 31)

Keep a large index fresh without re-embedding everything.

#	Pattern	Key idea	Notebook
31	Incremental RAG	SHA-256 page hashing — re-embed only changed pages	31_Incremental_RAG.ipynb

Incremental RAG — diff-and-update flow

flowchart TD
  DOC[New document version] --> HP[Hash each page SHA-256]
  HP --> DIFF{"Diff against manifest.json sidecar"}
  DIFF --> UNCH[Unchanged pages → skip entirely]
  DIFF --> CHG[Changed pages → delete old vectors, embed new chunks]
  DIFF --> NEW[New pages → embed and upsert]
  DIFF --> DEL[Removed pages → delete vectors]
  CHG & NEW --> UP[Upsert to index]
  UP --> MAN[Update manifest.json]

[Key Insight] NB 31 is fully idempotent — re-ingesting an unchanged document costs zero API calls. Chunk IDs are deterministic SHA-256 hashes of content+position, so upsert is safe to call repeatedly without creating duplicates.

Tier 9 — Multi-Tenant & Federation (NB 32–33)

Data isolation and cross-domain retrieval for SaaS and enterprise.

#	Pattern	Key idea	Notebook
32	Multi-Tenant RAG	Single shared collection, mandatory tenant_id payload filter on every query	32_Multi_Tenant_RAG.ipynb
33	Federated RAG	Parallel fan-out across domain collections + RRF merge + graceful failure	33_Federated_RAG.ipynb

Multi-Tenant RAG — isolation flow

flowchart TD
  ING[Ingest document for tenant T] --> TID[Stamp tenant_id on every chunk]
  TID --> CID["chunk_id = SHA-256 tenant+doc+page+text — prevents cross-tenant collisions"]
  CID --> IDX[(Shared collection with tenant_id payload index)]
  Q["Query from tenant T"] --> FILT["Mandatory filter: tenant_id = T on every ANN search"]
  FILT --> SEARCH[Scoped kNN — T never sees other tenants' data]
  IDX --> SEARCH
  SEARCH --> G[LLM Generate]

Federated RAG — cross-domain flow

flowchart TD
  Q[Question] --> RT["LLM router: select relevant collections clinical / technical / regulatory"]
  RT --> PAR["Parallel fan-out via thread pool timeout=12s per federate"]
  PAR --> C1[Clinical collection] & C2[Technical collection] & C3[Regulatory collection]
  C1 & C2 & C3 --> FAIL[Handle failures gracefully — non-fatal per federate]
  FAIL --> RRF["RRF merge with chunk_hash dedup — same passage from 2 feds appears once"]
  RRF --> TOP["Top-K with fed_id source attribution"]
  TOP --> G[LLM Generate with citations]

Pattern selection guide

flowchart TD
  START[What is your bottleneck?] --> A[Retrieval misses relevant chunks]
  START --> B[Answer quality is poor]
  START --> C[Latency or cost too high]
  START --> D[Multi-turn conversation]
  START --> E[Production scale]

  A --> A1["Tier 1: fix chunking — semantic, hierarchical, contextual"]
  A --> A2["Tier 2: hybrid search or reranking"]
  A --> A3["Tier 3: query decomposition or RAG fusion"]

  B --> B1["Tier 4: corrective RAG or self RAG"]
  B --> B2["Tier 3: CoT RAG for multi-step reasoning"]

  C --> C1["Tier 6: adaptive RAG — route to one strategy, not all"]
  C --> C2["Tier 7: caching RAG — skip repeated queries"]
  C --> C3["Tier 8: incremental — skip unchanged pages on re-index"]

  D --> D1["Tier 5: memory-augmented or multi-turn RAG"]

  E --> E1["Tier 7: streaming + evaluation + multi-modal pipeline"]
  E --> E2["Tier 9: multi-tenant or federated"]

Full pattern index

Tier	#	Pattern	Notebook
1 — Chunking	01	Simple RAG (baseline)	Link
1	02	Semantic Chunking	Link
1	03	Hierarchical RAG	Link
1	04	Parent-Child RAG	Link
1	05	Sentence Window RAG	Link
1	06	Contextual Retrieval	Link
2 — Retrieval	07	Hybrid Search (BM25 + Vector + RRF)	Link
2	08	HyDE	Link
2	09	Reranking	Link
2	10	Contextual Compression	Link
2	11	Metadata Filtering	Link
2	12	Multi-Document RAG	Link
3 — Query	13	Query Decomposition	Link
3	14	Step-Back Prompting	Link
3	15	RAG Fusion	Link
3	16	Chain-of-Thought RAG	Link
3	17	ReAct RAG	Link
4 — Agentic	18	Corrective RAG (CRAG)	Link
4	19	Self RAG	Link
4	20	Iterative RAG	Link
4	21	Recursive RAG	Link
4	22	Agentic RAG (Strands)	Link
5 — Memory	23	Memory-Augmented RAG	Link
5	24	Multi-Turn Conversational RAG	Link
6 — Ensemble	25	Ensemble RAG	Link
6	26	Adaptive RAG	Link
7 — Production	27	Streaming RAG	Link
7	28	Caching RAG	Link
7	29	Evaluation RAG	Link
7	30	Complete Pipeline (multi-modal)	Link
8 — Incremental	31	Incremental RAG	Link
9 — Multi-Tenant	32	Multi-Tenant RAG	Link
9	33	Federated RAG	Link

Full repository: github.com/Ramu-DE/rag-patterns-aws-Qdrant