Similarity With Dot Product
The one operation behind embeddings, search, and attention.
💡 The intuition
Two people rate pizza, hiking, and jazz. Multiply their ratings category by category and add it all up. Shared loves make the total big; opposite tastes make it small. That single number is the dot product — a score for how aligned two vectors are.
Query concept
king = [0.9, 0.8, 0.1]
Most similar to “king” (by cosine)
Step by step: king · queen
dot = 0.9×0.8 + 0.8×0.9 + 0.1×0.15 = 1.455
|king| = 1.208 , |queen| = 1.213
cosine = dot / (|king|·|queen|) = 0.992
Cosine near 1 = same direction (very similar); near 0 = unrelated; toward −1 = opposite.
A tiny worked example
A = [1, 2, 3], B = [2, 0, 4]
dot = (1×2) + (2×0) + (3×4) = 2 + 0 + 12 = 14
|A| = √14 ≈ 3.742, |B| = √20 ≈ 4.472
cosine = 14 / (3.742 × 4.472) ≈ 0.837 → close to 1 → similar direction
Free lesson complete
Continue with the complete Core AI Intuitions path.
Pro unlocks the surrounding lessons, runnable practice, and the full build-ready sequence.
Why it matters in AI
- ▸Semantic search ranks documents by cosine similarity to your query.
- ▸Attention scores are query · key — how much one token should attend to another.
- ▸Deduplication and clustering use cosine ≈ 1 to find near-identical items.
- ▸Recommendations match a user vector against item vectors with a dot product.
Things people get wrong
✗ A bigger dot product always means more similar.
✓ Only if magnitudes are comparable. Cosine normalizes out length so direction is what counts.
✗ Cosine and dot product are the same thing.
✓ Cosine is the dot product of two length-1 vectors. They only match after normalizing.
✗ Similarity can't be negative.
✓ Cosine ranges from −1 to +1. 0 means unrelated; negative means opposing direction.
Related reading · Knowledge Lab