Beginner8 min read3 of 40

Semantic Chunking

Same as Simple RAG but with smarter scissors — splits where the topic actually changes instead of every 1000 characters.

Read first:Simple RAG

Overview

Property	Detail
Pattern	Semantic Chunking — Smarter Splitting
Level	2 (Better chunks)
Key Improvement	Splits at meaning boundaries, not character counts
Result	Fewer chunks (63 vs 97) but each is a complete thought
Notebook	02_Semantic_Chunking.ipynb

Architecture

flowchart TD
    A[PDF Document] --> B[Read Full Text]
    B --> C["Embed Every Sentence (Bedrock Titan, 1024 dims)"]
    C --> D["Compare Adjacent Sentence Embeddings"]
    D --> E{"Similarity Drop Below Threshold?"}
    E -->|Yes| F["BREAKPOINT — Topic Changed"]
    E -->|No| D
    F --> G["Split at Breakpoints"]
    G --> H["Chunk 1: Topic A"]
    G --> I["Chunk 2: Topic B"]
    G --> J["Chunk 3: Topic C"]
    H --> K[(Store in Qdrant)]
    I --> K
    J --> K
    K --> L[Query as Usual]

Simple RAG vs Semantic Chunking

Aspect	Simple RAG	Semantic Chunking
Split method	Every 1000 characters	Where meaning shifts
Chunks produced	97	63 (fewer but better)
Risk	Cuts mid-sentence	Never splits a coherent idea
Chunk quality	Variable — may mix topics	High — each chunk = one topic
Speed	Fast	Slightly slower (embeds every sentence first)

Test Output

Same PDF, same query:

Simple RAG:     97 chunks (some cut mid-thought)
Semantic:       63 chunks (each is a complete topic)

Result: Better retrieval quality with fewer, more coherent chunks.

How the Breakpoints Work

Sentence Pair	Cosine Similarity	Action
"Greenhouse gases trap heat" → "CO2 is the most common"	0.85	Same chunk (related)
"CO2 levels rose 40%" → "Policy changes in 2015 included"	0.28	SPLIT (topic changed)
"Paris Agreement set targets" → "Nations committed to reduce"	0.81	Same chunk (related)

When to Use

Scenario	Use Semantic Chunking?
Long documents with multiple topics	Yes
Reports, research papers, manuals	Yes
Short single-topic documents	No — simple chunking is fine
Need fastest possible indexing	No — extra embedding step

Tech Stack

Component	Technology
Embeddings	AWS Bedrock — Amazon Titan Embed Text v2 (1024 dims)
LLM	AWS Bedrock — Claude Sonnet
Vector DB	Qdrant Cloud (cosine similarity)
PDF Reader	PyMuPDF (fitz)
Framework	LangChain

Source

▸02_Semantic_Chunking.ipynb

Related Concepts

Simple RAG

The foundation — load a PDF, split into chunks, embed with Bedrock Titan, store in Qdrant, and retrieve by similarity.

Reranking

The quality filter — retrieves 10 docs, then a second model re-scores them. Two methods: fast cross-encoder or smarter LLM reranking.

Relevant Segment Extraction (RSE)

The context reconstructor — instead of returning scattered chunks, finds the best contiguous section of the document that answers the query.