Dot products
Two vectors in, one number out — and why that number keeps appearing wherever similarity or projection is being measured.
This lesson is part of a preview roadmap.
The engine around it is real — progress, notes and bookmarks all work — but the video has not been through the sourcing and review protocol a shipped lesson goes through. See the roadmap for what that means.
Dot products
The player loads from youtube-nocookie.com when you press play. Nothing reaches YouTube before that.
What you'll learn
- The dot product is the length of one vector times the length of the other's shadow on it.
- Its sign alone answers "do these point the same way?" — positive, zero and negative mean aligned, perpendicular and opposed.
- Projecting onto a line and multiplying by a one-row matrix are the same operation, which is why the dot product shows up in every layer.
The operation everything is built from
A dot product is the smallest useful thing you can do with two vectors: pair them up, multiply, add. What makes it worth a node of its own is the second description — it measures how much of one vector lies along the other — because that is the description that explains why it appears in similarity search, in attention scores, and in every single unit of a dense layer.
What to watch for
The duality argument in the back half is the part to slow down on. It shows that every linear function from vectors to numbers is a dot product with some fixed vector. That is the reason a row of a weight matrix and a direction in space are interchangeable ideas, and it is the cleanest available answer to “why are neural networks matrix multiplications?”.
Where this sits
Stack enough dot products side by side and you have matrix multiplication, which is the next node. Much later, the same operation returns as the score between a query and a key.
Check yourself
Two unit vectors have a dot product of 0. What do you know?
They are perpendicular. With unit length the dot product is exactly the cosine of the angle between them, and cos 90° is 0.
Why is the dot product the natural measure of similarity for embeddings?
Because it grows with alignment and ignores nothing else — two embeddings pointing the same way score high however they were produced.
Go deeper
The maths underneath
Notes
Notes are saved to your account.
Everything else on this page works without one — progress and bookmarks keep themselves in this browser and come with you when you sign in. Notes are the one thing worth not trusting to a browser you might clear.
Your notes could not be loaded.