Overview
| Property | Detail |
|---|---|
| Pattern | Reliable RAG — The Trust-But-Verify System |
| Level | 5 (Verification + trust) |
| Key Innovation | 3 verification layers before returning an answer |
| LLM Calls | 4+ (grade + generate + hallucination check + source highlight) |
| Notebook | 18_Corrective_RAG.ipynb |
Architecture
flowchart TD
A[Question] --> B["1. RETRIEVE — Get 4 docs from Qdrant"]
B --> C["2. GRADE RELEVANCE\n(Is each doc relevant? yes/no)"]
C --> D["Relevant Docs Only"]
C --> X["Irrelevant — Discard"]
D --> E["3. GENERATE — Answer using only relevant docs"]
E --> F["4. HALLUCINATION CHECK\n(Is answer supported by docs?)"]
F -->|Grounded| G["5. HIGHLIGHT SOURCES\n(Which exact sentences were used?)"]
F -->|Not Grounded| H[Regenerate or Flag]
G --> I["FINAL OUTPUT\nAnswer + Grounded Status + Source Quotes"]Test Results (Actual Notebook Output)
Question: "What are the main causes of climate change?"
| Step | What Happened |
|---|---|
| ① Retrieve | Got 4 docs from Qdrant |
| ② Grade | Doc 1: ✓, Doc 2: ✓, Doc 3: ✗ (irrelevant), Doc 4: ✓ |
| ③ Generate | "Greenhouse gases, fossil fuels, deforestation..." |
| ④ Hallucination check | YES — answer is grounded in the docs |
| ⑤ Sources | 5 exact text segments identified and highlighted |
What Reliable RAG Answers That Simple RAG Cannot
| Question | Simple RAG | Reliable RAG |
|---|---|---|
| "Were the retrieved docs relevant?" | Unknown | Yes — graded each one |
| "Did the LLM make stuff up?" | Unknown | No — hallucination check passed |
| "Which source supports this claim?" | Unknown | Exact segments identified |
| "Can I trust this answer?" | Hope so | Verified and sourced |
The Tradeoff
| Aspect | Simple RAG | Reliable RAG |
|---|---|---|
| LLM calls | 1 | 4+ |
| Speed | Fast | Slower (4x more calls) |
| Cost | Low | ~4x higher |
| Trust level | "Hope it's right" | "Verified and sourced" |
| Audit trail | None | Full source tracing |
When to Use
| Scenario | Use Reliable RAG? |
|---|---|
| Legal / medical / financial apps | Yes — wrong answers have consequences |
| Audit trails required | Yes — shows exact sources |
| Building user trust | Yes — "here's exactly where this came from" |
| Internal quick lookups | No — overkill |
| Latency-sensitive chatbot | No — too slow |
Tech Stack
| Component | Technology |
|---|---|
| Embeddings | AWS Bedrock — Amazon Titan Embed Text v2 (1024 dims) |
| LLM | AWS Bedrock — Claude Sonnet |
| Vector DB | Qdrant Cloud (cosine similarity) |
| Chunk Size | 500 characters |
| Framework | LangChain |