Vector databases and embeddings: Visual walkthrough

This is the visual companion to the parent topic. If you want the same in Hinglish, see 4.4.12 Vector databases and embeddings (Hinglish).

Step 1 — What a vector actually is (an arrow with an address)

WHAT. A vector is just a list of numbers. We write $a = (a_{1}, a_{2})$ . But that list has a geometric meaning: start at the origin (the point $(0, 0)$ ), walk $a_{1}$ to the right, then $a_{2}$ up. The arrow from origin to where you land is the vector.

WHY we care. Embeddings (from the parent note) are vectors with hundreds of numbers. We can't draw 768 dimensions, but every idea we need already shows up in 2 dimensions, so we'll live there. The truth generalizes: nothing below secretly needs the third number.

PICTURE. Two arrows from the origin. Their lengths differ; their directions differ. Those are the only two things an arrow has, and the whole page is about separating them.

Step 2 — Length of an arrow (the norm) via one right triangle

WHAT. The length of $a = (a_{1}, a_{2})$ is written $∥ a ∥$ (two bars, read "the norm of a"). We compute it with Pythagoras: $∥ a ∥ = horizontal leg^{2} a_{1}^{2} + vertical leg^{2} a_{2}^{2}$

WHY this tool. Why Pythagoras and not something else? Because the horizontal step $a_{1}$ and the vertical step $a_{2}$ meet at a right angle — walk right, then turn 90° and walk up. Any right triangle obeys $leg^{2} + leg^{2} = hypotenuse^{2}$ , and the hypotenuse is our arrow. So Pythagoras is not a choice; it's forced by how we built the arrow.

PICTURE. The arrow is the slanted hypotenuse; the two components are the flat and vertical legs. The little square marks the right angle.

See Vector Norms for the general $d$ -dimensional version — the rule is identical, just more squares added under the root.

Step 3 — Multiplying two arrows: the dot product

WHAT. The dot product of $a = (a_{1}, a_{2})$ and $b = (b_{1}, b_{2})$ is one number: $a \cdot b = match x with x a_{1} b_{1} + match y with y a_{2} b_{2}$ Multiply matched components, then add. The result is a single scalar (plain number), not an arrow.

WHY define this at all? We are hunting for a number that says "how aligned are these two arrows?" The dot product is the simplest combination that grows when the arrows point the same way and shrinks (goes negative) when they point opposite. Step 4 will reveal exactly how it encodes the angle — that's the payoff.

PICTURE. Line up the two lists, multiply straight down each column, add across.

Step 4 — The hidden angle: dot product = lengths × cosine

WHAT. There is a second, geometric face of the same dot product: $a \cdot b = ∥ a ∥ ∥ b ∥ cos θ$ Here $θ$ (theta) is the angle between the two arrows, and $cos θ$ (cosine of that angle) is a number that answers: "how much do the two directions overlap?"

WHY cosine and not the angle itself? Because we need a number, and raw degrees don't multiply cleanly with lengths. Cosine is the exact function that converts "angle between arrows" into "fraction of overlap." Watch what it does:

Arrows pointing the same way → $θ = 0°$ → $cos 0° = 1$ (full overlap).
Arrows at a right angle → $θ = 90°$ → $cos 90° = 0$ (no overlap).
Arrows pointing opposite ways → $θ = 180°$ → $cos 180° = - 1$ (maximal disagreement).

So $cos θ$ slides smoothly from $1$ down to $- 1$ as the arrows swing apart. That is precisely the "same-meaning to opposite-meaning" scale we want for embeddings.

PICTURE. Same two arrows, now with the angle $θ$ marked between them, and a dial showing $cos θ$ at $1$ , $0$ , $- 1$ for the three landmark angles.

Step 5 — The problem: raw dot product is polluted by length

WHAT. Look at the geometric face again: $a \cdot b = ∥ a ∥∥ b ∥ cos θ$ . The number we get depends on three things: both lengths and the angle. But for meaning, we only care about the angle.

WHY this is a real problem. In the parent note: a long document about dogs and a short note about dogs should count as equally "about dogs." Same direction, different lengths. If we used raw dot product, the long document would score higher just for being long — length would drown out meaning.

PICTURE. Two arrows pointing the exact same way (angle $0$ ), one long, one short. Their dot product with a query differs wildly, even though their direction — their meaning — is identical.

Step 6 — The fix: divide out both lengths → cosine similarity

WHAT. Take the geometric identity and divide both sides by $∥ a ∥∥ b ∥$ : $\frac{a \cdot b}{∥ a ∥ ∥ b ∥} = \frac{∥ a ∥ ∥ b ∥ c o s θ}{∥ a ∥ ∥ b ∥} = cos θ$ The two length factors cancel on the right, leaving only $cos θ$ . This survivor is cosine similarity.

WHY divide, specifically, by the product of norms? Because that is the only thing standing between us and a pure angle. The identity literally packages the answer as $(lengths) \times (cos θ)$ ; to isolate $cos θ$ you divide by the lengths. No other operation removes both length factors at once.

PICTURE. The fraction with the two $∥ a ∥∥ b ∥$ factors struck out top and bottom, leaving a clean $cos θ$ — and the same two arrows rescaled to length $1$ so only their directions remain.

Step 7 — Every case, on one number line

WHAT. Because $cos θ$ ranges over $[- 1, 1]$ , cosine similarity can never escape that band. Let's nail down every region so no query ever surprises you.

Angle $θ$	$cos θ$	Meaning
$0°$	$1$	identical direction — same meaning
between $0°$ and $90°$	between $1$ and $0$	related, "leaning together"
$90°$	$0$	orthogonal — unrelated
between $90°$ and $180°$	between $0$ and $- 1$	leaning apart
$180°$	$- 1$	opposite direction

WHY cover the negatives too? Embedding models can produce vectors that point away from each other; a search must know that $- 0.3$ means "actively unlike," not just "a bit less similar." Retrieval keeps the largest similarity (closest to $+ 1$ ).

PICTURE. A single horizontal scale from $- 1$ to $+ 1$ with sample arrow-pairs drawn above each region.

Step 8 — The degenerate case: the zero vector

WHAT. What if $a = (0, 0)$ ? Then $∥ a ∥ = 0$ , and cosine similarity has a $0$ in the denominator — division by zero, undefined.

WHY it matters and how it happens. A zero vector is an arrow with no direction — it points nowhere, so "the angle between it and $b$ " is a meaningless question. In practice you get zero (or near-zero) vectors from empty chunks or degenerate embeddings. The picture makes the failure obvious: there is no arrow to measure an angle against.

PICTURE. A lone $b$ and, at the origin, a dot labelled "zero vector — no direction," with the undefined fraction beside it.

Step 9 — The shortcut databases use: pre-normalize, then just dot

WHAT. To normalize a vector means to shrink or stretch it to length exactly $1$ while keeping its direction: $\overset{a}{^} = \frac{a}{∥ a ∥}$ . After this, $∥ \overset{a}{^} ∥ = 1$ .

WHY this speeds up billions of searches. If every stored vector already has length $1$ , then in the cosine formula the denominator $∥ \overset{a}{^} ∥∥ \hat{b} ∥ = 1 \cdot 1 = 1$ , so $sim (\overset{a}{^}, \hat{b}) = \overset{a}{^} \cdot \hat{b}$ The expensive divisions vanish — the database only ever computes the cheap dot product (Step 3). Do the normalization once at insert time, reuse it for every future query.

PICTURE. Two long arrows collapsing onto the unit circle (radius-1 ring); once on the ring, only the angle between them remains, so a bare dot product already equals $cos θ$ .

The one-picture summary

Everything above, compressed: two raw arrows → measure lengths (Step 2) → dot them (Step 3) → that dot secretly equals lengths × cosine (Step 4) → length pollutes meaning (Step 5) → divide it out (Step 6) → land on the $[- 1, 1]$ dial (Step 7), minding the zero-vector hole (Step 8) → or just pre-normalize and dot (Step 9).

Recall Feynman: tell it back in plain words

Every sentence becomes an arrow from the centre of a map. An arrow has only two honest facts: how long it is and which way it points. For meaning, only the direction matters — a chatty long paragraph about dogs and a short line about dogs point the same way. To compare two arrows I could just multiply-and-add their numbers (the dot product), but that answer gets bigger just because an arrow is longer, which is cheating. So I divide by both arrows' lengths, and all that survives is $cos θ$ — a single number from $+ 1$ (same direction, same meaning) through $0$ (unrelated) down to $- 1$ (opposite). If an arrow has zero length it points nowhere, so the question is meaningless — I skip it. And if I shrink every arrow to length $1$ ahead of time, the dividing is already done, so plain multiply-and-add gives the cosine straight away — which is exactly the trick a vector database uses to search a billion arrows fast.

Recall Quick self-check

Why divide the dot product by both norms? ::: To cancel the two length factors in $∥ a ∥∥ b ∥ cos θ$ , leaving only $cos θ$ — pure direction. What is cosine similarity of two identical-direction vectors of different length? ::: $1$ — direction is the same, length is divided out. What does $cos θ = 0$ mean geometrically and semantically? ::: Arrows at $90°$ (orthogonal); unrelated meaning. Why can a database use a bare dot product for cosine? ::: Vectors are pre-normalized to length $1$ , so the denominator is $1$ . What breaks cosine similarity, and why? ::: A zero vector — length $0$ gives division by zero and has no direction to measure an angle from.

Connections

Vector Databases and Embeddings — the parent topic this walkthrough serves.
Dot Product · Vector Norms · Cosine Similarity — the three tools built above.
HNSW and ANN Indexes — why pre-normalized dot products (Step 9) make search fast.
Retrieval-Augmented Generation — the search this similarity powers.
Curse of Dimensionality — what changes when $d$ grows huge.