SPEC: Guardrails
Definition
[Definition] Guardrails are safety and control mechanisms — typically applied at the application layer, around an LLM — that detect, block, or filter unsafe, inappropriate, off-topic, or non-compliant inputs and outputs. They act as a protective layer on top of the model's own alignment training, enforcing developer-defined policies.
Why Guardrails Are Needed
Even well-aligned LLMs can:
- ▸Generate harmful content under adversarial prompts
- ▸Drift out of scope (medical chatbot discussing politics)
- ▸Produce PII or confidential data
- ▸Generate content that violates legal/regulatory requirements
- ▸Be manipulated by prompt injection
[Key Insight] Guardrails enforce these boundaries reliably at the application level.
Guardrail Layers
Guardrails operate at two points in the LLM pipeline:
Input Guardrails (Pre-generation)
Applied to the user's input before it reaches the LLM:
- ▸Block jailbreak attempts
- ▸Detect harmful intent (violence, self-harm, illegal activity)
- ▸Filter prompt injection attacks
- ▸Enforce topic scope ("this chatbot only discusses our products")
- ▸PII detection (block or redact personal data before sending to LLM)
- ▸Language filtering
Output Guardrails (Post-generation)
Applied to the LLM's response before it reaches the user:
- ▸Toxicity/hate speech detection
- ▸PII detection and redaction
- ▸Off-topic response filtering
- ▸Hallucination detection
- ▸Competitor mention detection
- ▸Fact verification
Guardrail Implementation Methods
Rule-Based
- ▸Regular expressions for pattern matching (credit card numbers, phone numbers)
- ▸Keyword blocklists
- ▸Simple, fast, fully deterministic
- ▸Limited flexibility for nuanced cases
Classifier-Based
- ▸Small fine-tuned models trained to classify inputs/outputs
- ▸Examples: toxicity classifier, topic classifier, PII detector
- ▸More flexible than rules, slightly slower
- ▸Examples: Perspective API, Meta's Llama Guard
LLM-as-Judge
- ▸Use a second (often smaller) LLM to evaluate input/output
- ▸Prompt: "Does the following response contain harmful content? Yes/No"
- ▸More flexible and generalizable
- ▸Higher latency and cost
Embedding-Based
- ▸Embed input, compare to embeddings of known harmful patterns
- ▸Threshold-based similarity filtering
- ▸Fast but less precise for nuanced attacks
Guardrail Frameworks and Tools
| Tool | Type | Notes |
|---|---|---|
| NVIDIA NeMo Guardrails | Framework | Programmatic rails with LLM colang scripting |
| Llama Guard (Meta) | LLM classifier | Open-source safety classifier for inputs/outputs |
| OpenAI Moderation API | API | Toxicity/harm classification |
| AWS Bedrock Guardrails | Managed | Topic denial, PII, word filters, grounding |
| Azure Content Safety | Managed | Microsoft's content moderation API |
| Guardrails AI | Framework | Python library, validators, structured output |
| Rebuff | Framework | Prompt injection detection |
| LangChain callbacks | Framework | Custom logic at any pipeline step |
Common Guardrail Categories
Content Safety
- ▸Block: hate speech, violence, self-harm, CSAM
- ▸Method: classifier (Llama Guard, Perspective API, OpenAI Moderation)
Topic Scope Enforcement
- ▸Block: out-of-domain queries (a banking bot discussing recipes)
- ▸Method: topic classifier, semantic similarity to allowed topics
PII Protection
- ▸Detect/redact: names, SSNs, emails, phone numbers, credit card numbers
- ▸Method: NER models (spaCy, AWS Comprehend), regex rules
Prompt Injection Defense
- ▸Block: attempts to override system prompt, jailbreaks, role-playing attacks
- ▸Method: injection detector (Rebuff, custom classifier), system prompt hardening
Hallucination / Grounding Check
- ▸Verify: generated answer is supported by provided context
- ▸Method: NLI model, LLM-as-judge faithfulness check
Brand / Compliance
- ▸Block: competitor mentions, prohibited topics, off-brand language
- ▸Method: keyword lists + classifier
Guardrail Pipeline Design
User Input
↓
[Input Guardrail]
↓ (if safe)
[LLM Generation]
↓
[Output Guardrail]
↓ (if safe)
User Response
AWS Bedrock Guardrails (Example Managed Service)
- ▸Topic denial: block defined topics
- ▸Content filters: violence, hate, sexual, self-harm (adjustable thresholds)
- ▸Word filters: custom keyword blocklists
- ▸PII redaction: automatically redact/mask PII
- ▸Grounding check: verify response against retrieved context
- ▸Sensitive info filters: detect/redact custom regex patterns
Guardrail Trade-offs
| Trade-off | Description |
|---|---|
| Accuracy vs. latency | Better classifiers = higher latency |
| Precision vs. recall | Strict rails → false positives (blocking valid content) |
| Coverage vs. cost | More checks = higher cost per request |
| Rule rigidity vs. flexibility | Rules are fast but brittle; ML is flexible but slower |
Evaluation of Guardrails
| Metric | Description |
|---|---|
| False positive rate | Valid inputs incorrectly blocked |
| False negative rate | Harmful inputs that slipped through |
| Latency overhead | Added ms per request |
| Coverage | % of harm categories addressed |
Red-Teaming Guardrails
Test guardrails with adversarial inputs:
- ▸Known jailbreak patterns ("DAN", "ignore previous instructions")
- ▸Encoded attacks (Base64, ROT13, character substitution)
- ▸Indirect attacks (roleplay, hypothetical framing)
- ▸Multi-turn attacks (build up context over several turns)
Related Concepts
- ▸Alignment, Hallucination, Grounding, System Prompt, RLHF, Safety, Prompt Injection, RAG