Parent Document Retrieval - Fine Retrieve, Broad Context

Overview

Parent Document Retrieval resolves a fundamental chunking dilemma: small chunks give precise embedding matches but lose surrounding context; large chunks preserve context but embed poorly because one embedding must represent too many ideas. The solution is to index at two granularities — retrieve with the small child chunk, but return the larger parent chunk to the LLM.

Pattern #23 in the 37 RAG Patterns Collection

flowchart TD
  DOC[Document] --> L1[Level 1: Full document]
  DOC --> L2[Level 2: Sections]
  DOC --> L3[Level 3: Paragraphs]
  DOC --> L4[Level 4: Sentences]
  L4 --> EMB[Embed sentences — smallest unit]
  EMB --> IDX[(Index sentences with parent_id chain)]
  Q[Query] --> SE[Embed]
  SE --> SR[Search sentences top-K]
  IDX --> SR
  SR --> RP[Walk up tree to best parent level]
  RP --> LLM[Generate with rich context]

The Problem

Chunking involves a precision-context tradeoff:

Document: A 2,000-token API reference section

Option A — Small chunks (100 tokens):
  Retrieval precision:  HIGH  ✅  (embedding closely matches the query)
  Generation context:   LOW   ❌  (chunk lacks surrounding explanation)
  Risk: LLM answers from a fragment that's accurate but missing critical context

Option B — Large chunks (800 tokens):
  Retrieval precision:  LOW   ❌  (one embedding averages over many concepts)
  Generation context:   HIGH  ✅  (LLM has full section context)
  Risk: Vector search misses the right document because the embedding is diluted

Neither extreme is optimal. You want the precision of small chunks with the context richness of large chunks.

Solution

Index two sizes. Retrieve the small one; return the large one:

Indexing time:
  Parent chunk (500 tokens): stored in docstore, keyed by parent_id
  Child chunks (100 tokens each): embedded and stored in vector index,
                                   each carrying parent_id in metadata

Retrieval time:
  Query → vector search on child chunks → top-k children
  → look up parent_id for each child → fetch parent from docstore
  → deduplicate parents → send parents to LLM

Implementation

python

from typing import List, Dict
import uuid

# ── Indexing ──────────────────────────────────────────────────────────────────

def split_into_parents(document: str, parent_size: int = 500) -> List[Dict]:
    """Split a document into large parent chunks."""
    words = document.split()
    parents = []
    for i in range(0, len(words), parent_size):
        chunk = ' '.join(words[i: i + parent_size])
        parents.append({
            'id': str(uuid.uuid4()),
            'content': chunk,
        })
    return parents


def split_parent_into_children(parent: Dict, child_size: int = 100) -> List[Dict]:
    """Split one parent chunk into small child chunks, each pointing back to the parent."""
    words = parent['content'].split()
    children = []
    for i in range(0, len(words), child_size):
        chunk = ' '.join(words[i: i + child_size])
        children.append({
            'id': str(uuid.uuid4()),
            'content': chunk,
            'parent_id': parent['id'],   # key link back to parent
        })
    return children


def build_parent_child_index(documents: List[str], vector_store, doc_store):
    """
    Build the two-store index:
    - vector_store: holds child chunk embeddings
    - doc_store:    holds parent chunk text, keyed by parent_id
    """
    for doc_text in documents:
        parents = split_into_parents(doc_text, parent_size=500)

        for parent in parents:
            # Store parent in docstore
            doc_store[parent['id']] = parent['content']

            # Embed and store children in vector index
            children = split_parent_into_children(parent, child_size=100)
            for child in children:
                vector_store.add(child)   # embed child['content'], store with metadata


# ── Retrieval ─────────────────────────────────────────────────────────────────

def parent_document_retrieve(query: str, vector_store, doc_store, top_k: int = 5) -> List[Dict]:
    """
    Retrieve top-k child chunks, then return their deduplicated parent chunks.
    """
    # Step 1: Search child chunks (small, precise embeddings)
    child_results = vector_store.search(query, top_k=top_k * 3)  # over-fetch children

    # Step 2: Look up parent for each child
    parent_ids_seen = set()
    parent_chunks = []
    for child in child_results:
        pid = child['metadata']['parent_id']
        if pid not in parent_ids_seen:
            parent_ids_seen.add(pid)
            parent_content = doc_store.get(pid)
            if parent_content:
                parent_chunks.append({
                    'content': parent_content,
                    'parent_id': pid,
                    'matched_child': child['content'],
                })
        if len(parent_chunks) == top_k:
            break

    return parent_chunks


# ── Full RAG Pipeline ─────────────────────────────────────────────────────────

def parent_document_rag(query: str, vector_store, doc_store, llm, top_k: int = 5) -> Dict:
    """Parent Document RAG: retrieve with children, generate with parents."""
    # Step 1: Retrieve parent chunks via child-level search
    parents = parent_document_retrieve(query, vector_store, doc_store, top_k=top_k)

    # Step 2: Build context from full parent chunks
    context = "\n\n".join([p['content'] for p in parents])

    # Step 3: Generate answer
    answer = llm.invoke(f"Context:\n{context}\n\nQuestion: {query}\n\nAnswer:")

    return {
        'answer': answer,
        'parent_chunks_used': len(parents),
        'sources': [p['parent_id'] for p in parents],
    }

Complete Notebook

Run the full implementation:

Parent Document RAG AWS Notebook

Includes:

▸Two-store setup with Amazon OpenSearch and an in-memory docstore
▸Parent/child size tuning experiments (50/500, 100/500, 200/800 tokens)
▸Precision and context quality comparison against flat chunking
▸Integration with Amazon Bedrock embeddings

Performance

Metric	Standard Chunking	Parent Document
Precision@5	68%	81%
Context completeness	52%	89%
Avg. tokens/query	500	2,500
Answer quality (human eval)	71%	86%

Parent Document retrieval improves Precision@5 by +13 points and context completeness by +37 points, at the cost of more tokens sent to the LLM per query.

When to Use

Best For

▸Technical documentation — API references, specifications, and how-to guides where code snippets only make sense in their surrounding prose
▸Legal and compliance documents — clause-level retrieval that returns the full article or section
▸Books and long-form content — sentence-level search that surfaces the full chapter section to the LLM
▸Any corpus where section context dramatically changes the meaning of individual sentences

Combine With

▸Cross-Encoder Reranking (#4) — rerank the child-level results before resolving to parents, for better top-K selection
▸Contextual Compression (#6) — trim parent chunks to only the portions most relevant to the query before sending to the LLM

Next Steps

▸Try it: Run the notebook
▸Tune sizes: Experiment with child_size (50–200 tokens) and parent_size (400–800 tokens) for your corpus
▸Add compression: Layer Contextual Compression (#6) to trim large parent chunks before generation

Part of: 37 RAG Patterns Collection