Knowledge Lab
Master AI/ML from first principles
102 articles
What Is Vector Search?
How machines find meaning by turning words into coordinates — and why search changed forever when they did.
Embeddings: Turning Text into Vectors
What embedding models are, how they produce vectors, why context changes the output, and how to generate and cache embeddings efficiently.
Similarity Metrics: Cosine, Euclidean, Dot Product
How the three main distance functions work, when each one wins, and the key insight that cosine ignores magnitude — which is almost always what you want for text.
kNN and HNSW: How Vector Search Scales
Why brute-force kNN breaks at scale, how HNSW solves it with a multi-layer graph, and which three parameters control the accuracy vs. speed tradeoff.
Dense Vector Search: How Semantic Search Works
The full lifecycle of dense semantic search — from document ingestion through embedding, HNSW indexing, and query — with benchmarks on what it gets right and where it fails.
Sparse Encoding: Fast, Interpretable, and Underrated
TF-IDF sparse vectors explained — why 99.97% of the vector is zeros, how that makes search 10x faster than dense, and when sparse consistently beats semantic.
Hybrid Search: Combining Dense, Sparse, and Keyword
How combining three search engines — keyword, sparse, and dense — pushes NDCG from 0.85 to 0.91, with score normalization, weight tuning, and the architecture that makes it fast.
Vector Indexes: How Search Engines Store and Retrieve Vectors
How vector indexes work under the hood — collections, payload filtering, snapshot strategy, and the operational details that matter when serving real search traffic.
Quantization: Cut Vector Memory by 4–40x
INT8, FP16, and binary quantization explained — the memory-recall tradeoff, how rescoring recovers lost precision, and when compression actually hurts.
Measuring Search Quality: NDCG, Recall, and A/B Testing
How to measure search quality with NDCG@10 and recall@k, build a ground-truth dataset, benchmark your pipeline, and run A/B tests to validate improvements.
LLM (Large Language Model)
Transformer-based model trained on massive text corpora that generates text by predicting the next token.
Token
The atomic unit of text that an LLM processes — a sub-word piece that balances vocabulary size and semantic meaning.
Tokenization
Converting raw text into token IDs the model can process — and back again.
Embeddings
Dense numerical vectors that encode semantic meaning — similar concepts cluster together in high-dimensional space.
Latent Space
The high-dimensional space where embeddings live and organize by semantic meaning.
Parameters
The learned numerical weights that store everything the model knows — billions of numbers shaped by training.
Next Token Prediction
The core training objective: given prior tokens, predict the most probable next one.
Transformer Architecture
The neural architecture underlying all modern LLMs — processes all tokens in parallel using self-attention.
Attention / Self-Attention
The mechanism that lets every token dynamically focus on every other token based on relevance.
Scaling Laws
Empirical rules predicting LLM performance as a function of model size, data, and compute.
Logits & Softmax
Raw unnormalized scores from the model output layer, converted to probabilities via softmax.
Mixture of Experts (MoE)
Architecture where only a subset of parameters activate per token — enabling massive scale efficiently.
Pre-training
Training a model from scratch on massive text corpora using next-token prediction as the objective.
Base Model
The raw pre-trained model — powerful but not instruction-following. The foundation everything else builds on.
Instruct Model
A base model fine-tuned to follow instructions as a helpful assistant.
Fine-Tuning
Continued training on a smaller task-specific dataset to adapt the model for particular behaviors.
LoRA & PEFT
Parameter-Efficient Fine-Tuning — adapt only a tiny fraction of parameters (0.1–1%) rather than the full model.
Quantization
Reducing weight precision from FP32 to INT8/INT4 to shrink memory footprint and speed up inference.
Alignment
Training the model to be helpful, honest, and harmless — making model behavior match human values.
RLHF
Reinforcement Learning from Human Feedback — using human preference rankings to guide model behavior.
DPO (Direct Preference Optimization)
A simpler alternative to RLHF that directly optimizes from preference pairs without a reward model.
Guardrails
Safety mechanisms that block harmful or off-policy inputs and outputs in production systems.
Prompt Injection
Malicious input that overrides developer instructions to hijack model behavior.
Prompt
The full text input sent to the model — everything it sees before generating a response.
System Prompt
Developer-set instructions that define the model's role, rules, and behavior for a session.
User Prompt
The end user's actual question or instruction sent at runtime.
Zero-Shot Prompting
Asking the model to perform a task without any examples — relying purely on its training.
Few-Shot Prompting
Providing examples in the prompt to guide model behavior without changing model weights.
Chain of Thought
Prompting the model to show step-by-step reasoning before answering, dramatically improving accuracy on complex tasks.
In-Context Learning
The model's ability to adapt to new tasks purely from examples in the prompt — no weight updates needed.
Context Window
The maximum number of tokens the model can see and process at once — its working memory.
Inference
The process of generating output from a trained model — what happens when you call the API.
Latency
Time-to-first-token and total generation time — the key performance metrics for production LLM systems.
Temperature
The parameter controlling randomness in token sampling — 0 is deterministic, higher values are more creative.
Top-P & Top-K Sampling
Sampling strategies that filter which tokens can be selected as the next output token.
KV Cache
Cached attention key-value states enabling efficient autoregressive generation without recomputing past tokens.
Streaming
Delivering tokens incrementally as they are generated rather than waiting for the full response.
Reasoning Models
Models that use extended internal 'thinking' — visible chain-of-thought before answering — for complex tasks.
Hallucination
When the model confidently generates false, fabricated, or unsupported content as if it were fact.
Grounding
Constraining model outputs to provided, verifiable information to prevent hallucination.
RAG (Retrieval-Augmented Generation)
Retrieving relevant external documents at query time and injecting them into the prompt to ground generation.
Workflow
A fixed, predefined sequence of LLM-powered steps with deterministic control flow.
Agent
An LLM used as a dynamic reasoning engine that plans, uses tools, and adapts based on feedback.
Tool Use & Function Calling
The model outputs structured requests to invoke external tools — APIs, databases, code executors.
Structured Output
Constraining model output to a predefined schema — JSON, XML — for reliable downstream processing.
MCP (Model Context Protocol)
Anthropic's open protocol for connecting LLMs to external tools, data sources, and services.
Prompt Caching
Reusing precomputed KV states for repeated prompt prefixes to slash latency and cost.
Benchmarks
Standardized tests used to compare model capabilities across tasks and providers.
Evals
Custom evaluation frameworks for measuring model performance on your specific use case.
LLM-as-Judge
Using a powerful model to evaluate another model's outputs — scalable alternative to human evaluation.
Multimodality
Models that process and generate across text, images, audio, and video simultaneously.
Emergent Abilities
Capabilities that appear suddenly and unpredictably in models above certain scale thresholds.
33 RAG Patterns — Complete Tier Guide
33 production RAG patterns organized across 9 tiers — from basic chunking to multi-tenant federation. Each pattern links to a working notebook.
Simple RAG
The foundation — load a PDF, split into chunks, embed with Bedrock Titan, store in Qdrant, and retrieve by similarity.
Semantic Chunking
Same as Simple RAG but with smarter scissors — splits where the topic actually changes instead of every 1000 characters.
Hierarchical Indexing - Multi-Level Document Structure
Index documents at multiple granularities — summaries at the top, sections in the middle, sentences at the bottom — and retrieve at the right level for each query.
Parent Document Retrieval - Fine Retrieve, Broad Context
Index small child chunks for precise retrieval, but return their larger parent chunk to the LLM — combining retrieval precision with generation context.
Sentence Window Retrieval - Precise Match, Rich Context
Index individual sentences for precise embedding match, then expand each match to its surrounding N-sentence window before passing to the LLM.
Hybrid Search RAG - Best of All Worlds
Pattern #34 - Hybrid Search RAG - Best of All Worlds. Complete implementation with AWS Bedrock, code examples, and performance metrics.
HyDE - Hypothetical Document Embeddings
Generate hypothetical ideal answers, embed them, and use those embeddings to find better source documents. Improves retrieval for complex queries.
Reranking
The quality filter — retrieves 10 docs, then a second model re-scores them. Two methods: fast cross-encoder or smarter LLM reranking.
Cross-Encoder Reranking - Precise Relevance Scoring
Re-score retrieved candidates with a cross-encoder model that jointly encodes query and document for highly accurate relevance ranking.
Contextual Compression - Smart Context Reduction
Intelligently compress retrieved context by removing irrelevant information while preserving key facts. Reduces tokens and improves response quality.
Query Decomposition - Break Complex Queries
Pattern #9 - Query Decomposition - Break Complex Queries. Complete implementation with AWS Bedrock, code examples, and performance metrics.
Step-Back Prompting - Abstract Before You Search
Transform a specific, narrow query into a broader abstract question first, retrieve general principles, then combine abstract and specific context for a grounded answer.
Fusion Retrieval - Multi-Strategy Search
Combine multiple retrieval strategies (vector, keyword, graph) with reciprocal rank fusion for robust search results across diverse query types.
Multi-Query RAG - Parallel Retrieval from Multiple Angles
Generate several distinct sub-queries from a complex question, retrieve for each in parallel, then merge and deduplicate before generation.
Query Routing - Send Each Query to the Right Retriever
Classify each incoming query and route it to the most appropriate retrieval backend — vector store, SQL, graph, or web — instead of always using the same index.
Corrective RAG (CRAG) - Self-Correcting Retrieval
Automatically detect and correct irrelevant or incorrect retrievals. Uses relevance grading and web search fallback for improved accuracy.
Self-RAG
The smart decision maker — decides if retrieval is needed, filters irrelevant docs, generates multiple answers, and picks the best one.
Iterative Retrieval - Progressive Query Refinement
Retrieve, inspect intermediate results, refine the query, retrieve again — iterate until the retrieved context is sufficient to answer the question.
Recursive RAG - Iterative Deep Search
Pattern #10 - Recursive RAG - Iterative Deep Search. Complete implementation with AWS Bedrock, code examples, and performance metrics.
Agentic RAG - Autonomous Decision Making
Empower LLMs to autonomously decide when to retrieve, what to search, and how to combine information. Advanced pattern for complex multi-step reasoning.
Memory Augmented RAG - Conversational Context Across Turns
Persist conversation history and user context across turns so retrieval is conditioned on the full dialogue, not just the latest query.
Ensemble Retrieval - Combining Multiple Retrieval Strategies
Run multiple retrieval strategies in parallel and merge results using Reciprocal Rank Fusion for higher recall than any single retriever.
Adaptive RAG - Dynamic Strategy Selection
Pattern #8 - Adaptive RAG - Dynamic Strategy Selection. Complete implementation with AWS Bedrock, code examples, and performance metrics.
Streaming RAG - Real-Time Responses
Pattern #32 - Streaming RAG - Real-Time Responses. Complete implementation with AWS Bedrock, code examples, and performance metrics.
Cached RAG - Performance Optimization
Pattern #33 - Cached RAG - Performance Optimization. Complete implementation with AWS Bedrock, code examples, and performance metrics.
Production RAG - Enterprise Deployment
Pattern #35 - Production RAG - Enterprise Deployment. Complete implementation with AWS Bedrock, code examples, and performance metrics.
RAPTOR
Recursive Abstractive Processing for Tree-Organized Retrieval — builds a hierarchical summary tree for multi-level querying.
Graph RAG - Knowledge Graph Enhanced Retrieval
Enhance RAG with knowledge graphs to capture entity relationships and improve contextual understanding. Learn how to build graph-based retrieval systems.
Tree of Thoughts RAG - Branching Reasoning
Pattern #15 - Tree of Thoughts RAG - Branching Reasoning. Complete implementation with AWS Bedrock, code examples, and performance metrics.
Knowledge Graph RAG - Structured Reasoning with Entity Graphs
Augment RAG with a knowledge graph of entities and relationships for multi-hop reasoning, precise entity lookups, and explainable answers.
Multimodal RAG - Text, Images & Beyond
Pattern #11 - Multimodal RAG - Text, Images & Beyond. Complete implementation with AWS Bedrock, code examples, and performance metrics.
Hypothetical Questions - Index by Questions, Not Answers
At index time, generate hypothetical questions each chunk could answer. At query time, match the user query to these questions for dramatically better semantic alignment.
Auto-Merging Retrieval - Reconstruct Context from Overlapping Chunks
When enough child chunks from the same parent are retrieved, automatically merge them back into the parent chunk for richer, coherent context.
RAG Fusion - Multi-Query Generation with Reciprocal Rank Fusion
Generate multiple query variants from a single question, retrieve for each, then fuse ranked results with RRF to surface the most relevant documents.
Reliable RAG
The trust-but-verify system — adds 3 verification layers: relevance grading, hallucination detection, and source highlighting.
Retrieval with Feedback Loop
The learning system — collects user feedback after each answer and uses it to improve future retrievals. Gets smarter the more you use it.
Relevant Segment Extraction (RSE)
The context reconstructor — instead of returning scattered chunks, finds the best contiguous section of the document that answers the query.
Recursive Summarization - Hierarchical Document Compression
Recursively summarize a document bottom-up — chunk summaries → section summaries → document summary — creating a compression tree for efficient retrieval at any level.
Citation RAG - Grounded Answers with Source Attribution
Force the LLM to cite specific source chunks inline while generating, making every claim verifiable and reducing hallucination by anchoring to retrieved evidence.