Knowledge Graph RAG
Some questions have no single passage that answers them — the answer is stitched across documents, living in the relationships between entities. This is where a knowledge graph does what a vector index can't.
🕸️ One question, three documents
Imagine a compliance analyst with a few thousand contracts, amendments, and memos, asking: “Which of our obligations are exposed if this one milestone slips?” The answer isn't in any single document — it's stitched across three: a master agreement tying a payment to a milestone, an amendment that reschedules that milestone, and a risk memo tracing the knock-on effect. Keyword search returns the paragraphs that mention “milestone”; it can't follow the thread between them.
Where vector search runs out of road
Vector-only RAG retrieves by embedding similarity, so it judges each chunk in isolation. That is fine for “What is the warranty period in section 4?” but it struggles with questions whose answers are distributed or relational:
Multi-hop
“Which obligations depend on a milestone this amendment changed?” The reasoning chain spans documents that never share similar wording.
Cross-document aggregation
“Which clauses across all contracts reference the same indemnity cap?” No single chunk contains the answer.
Global / thematic
“Summarize the main themes and risks across the entire corpus.” There is no nearest chunk — the answer is a synthesis over everything.
Top-k nearest-neighbor retrieval returns locally similar text, not the connective structure that actually answers these questions. That structure is exactly what a knowledge graph captures.
Vector search sees isolated chunks
Each chunk is judged on its own by similarity. Nothing connects them — so a chain of reasoning that lives between documents is invisible.
A graph sees entities and relationships
The contract, its amendment, and the risk memo are linked through shared entities — so “what depends on what” becomes a path you can follow.
Which retrieval fits your question?
Pick the shape of your question — the right approach depends on it.
“Which obligations depend on a milestone this amendment changed?”
The reasoning chain spans documents that never share similar wording. Start from the entities the question names, then walk outward along their relationships from one document to the next.
Turning a corpus into a graph
Before any question is asked, the documents are turned into a graph of entities and their relationships. Grouped into three phases:
Prepare
Parse, load, and chunk the raw documents — and optionally translate them to one language so extraction is consistent.
Build the graph
An LLM reads each chunk and extracts entities and the relationships between them; a second pass catches missed items and merges duplicate entities that refer to the same thing.
Analyze & index
Community detection groups tightly-connected entities into topics, optional summaries are written per topic, and everything is written to a graph store plus lexical and vector indices.
Two ways to reason over the graph
Community summarization
Spend the expensive reasoning once, at indexing time, so queries stay cheap. Detect tightly-connected clusters of entities — topics that fall out of how the corpus actually hangs together — then write a summary of each, and of each cluster above it, producing a tree of thematic summaries at several zoom levels.
- • Global questions fan out across the summaries and reduce many partial answers into one.
- • Local questions start from named entities and walk outward along their relationships.
Dual-level keyword retrieval
Keep indexing cheap and do the thinking when the question arrives. Extract the graph, then stop — no summaries. At query time, pull two kinds of keyword from the question and send each to a different layer of the graph:
- • Low-level keywords (“obligation”, “milestone”) match entity descriptions — the “what things are” layer.
- • High-level keywords (“exposure”, “dependency”) match relationship descriptions — the “how things connect” layer.
Two opposite bets: pay the reasoning cost once at indexing time, or pay it on every query. Which is right depends on your corpus and your question mix — worth testing both rather than reasoning about it in the abstract.
The retrieval backbone: three retrievers, one pipeline
However you traverse the graph, every query fuses three kinds of retrieval into one ranked context — combined with Reciprocal Rank Fusion, optionally reranked, then used to generate a grounded answer with source attributions.
Lexical (BM25)
Exact term matching — catches SKUs, codes, and precise wording.
Semantic (kNN)
Embedding similarity — catches meaning, synonyms, and paraphrase.
Structural (graph)
Relationship expansion — follows the connections between entities across documents.
Only the graph traversal changes between approaches; the fusion, rerank, and generation steps stay identical — which is what makes switching between them a configuration change rather than a re-index.
Related reading · Knowledge Lab