SPEC: Grounding

Definition

[Definition] Grounding is the practice of constraining an LLM's outputs to provided, verifiable information — "grounding" the model's responses in a factual foundation rather than allowing it to rely purely on potentially incorrect parametric memory. A grounded model answers based on supplied evidence, not imagination.

The Grounding Problem

LLMs have two sources of "knowledge":

▸Parametric knowledge — baked into model weights during training (can be wrong/outdated)
▸Contextual knowledge — explicitly provided in the prompt (can be controlled and verified)

[Key Insight] Grounding means instructing the model to use (2) and not (1) — or to explicitly signal when (2) is insufficient.

Types of Grounding

Document Grounding

▸Provide the source document(s) in the prompt
▸Instruct: "Answer only using the information in the document below"
▸Use case: Q&A over a contract, policy, or technical manual

Data Grounding

▸Provide structured data (tables, JSON, SQL results) in the prompt
▸Model reasons over the provided data rather than inventing numbers
▸Use case: financial analysis, database query interpretation

Tool/Search Grounding

▸Give the model access to real-time search or APIs
▸Model retrieves current information before answering
▸Use case: questions about recent events, current prices, live data
▸Examples: Bing plugin (ChatGPT), Google Search (Gemini), web search tools

Retrieval Grounding (RAG)

▸Automatically retrieve relevant documents from a knowledge base at query time
▸Inject retrieved chunks into the prompt
▸Most scalable approach for large document collections

Citation-Based Grounding

▸Require the model to cite the specific passage supporting each claim
▸Enables human verification of every generated statement
▸Common in enterprise document workflows

Grounding Instructions in Prompts

System: You are a document analyst. Answer questions ONLY based on the 
document provided below. If the answer is not found in the document, 
respond with: "This information is not available in the provided document."
Do not use outside knowledge.

Document: [document content]

User: [question]

Grounding Verification Pipeline

After generation, verify grounding automatically:

1. LLM generates response with citations
2. Entailment model checks: does the document support each claim?
3. Claims not supported by any source → flagged or removed
4. Grounded claims → passed to user

Tools: NLI (Natural Language Inference) models, LLM-as-judge patterns

Grounding vs. Hallucination

Concept	Relationship
Hallucination	What happens when grounding fails
Grounding	The technique to prevent hallucination
RAG	The primary architecture for scalable grounding

Grounding Quality Metrics

Metric	Description
Faithfulness	% of claims supported by provided context
Relevance	% of context actually used in the answer
Attribution accuracy	Are citations correct?
Groundedness score	Composite measure from RAGAS, TruLens

Grounding in RAG Pipelines

The full grounded RAG pipeline:

User query 
  → [Retriever] → relevant document chunks
  → [LLM Prompt] = system + chunks + query
  → [LLM] → response grounded in retrieved chunks
  → [Verifier] → check faithfulness (optional)
  → User receives grounded answer + citations

Grounding Challenges

Context Faithfulness Failure

▸Model has retrieved context but ignores it
▸Reverts to parametric memory anyway
▸Fix: stronger grounding instructions, lower temperature, context window positioning

Retrieval Quality

▸If the wrong chunks are retrieved, the model is grounded in the wrong information
▸Grounding only works as well as the retrieval component

Context Conflicts

▸Retrieved document contradicts model's parametric knowledge
▸Model may blend both, producing a partially grounded answer
▸Fix: explicit instruction to prioritize provided context

No-Answer Handling

▸When the answer isn't in the provided context, model should say so
▸Instead, it may hallucinate an answer
▸Fix: explicit "say I don't know" instruction + output classification

Real-World Grounding Applications

Application	Grounding Technique
Customer support chatbot	Product documentation RAG
Legal document review	Document + citation grounding
Medical information assistant	Clinical guidelines RAG
Financial report analysis	Structured data grounding
Enterprise search	Internal knowledge base RAG
Coding assistant	API documentation grounding

Related Concepts

▸RAG, Hallucination, Context Window, Retrieval, Citations, Faithfulness, System Prompt