Reading and reproducing ML papers: Visual walkthrough

We build every idea from scratch. If you have never seen a dot product, a variance, or a softmax — good. You will meet each one at the exact moment we need it, and never before.

Step 1 — What is a "query" and a "key"? Two arrows.

WHAT. In the Transformer, every word is turned into a list of numbers — an arrow (a vector) in space. When we ask "how much should word A pay attention to word B?" we compare two arrows:

$Q$ (the query) = "what am I looking for?"
$K$ (the key) = "what do I offer?"

Each arrow has $d_{k}$ numbers in it. That letter $d_{k}$ just means "how many numbers are in each arrow" — its dimension. If $d_{k} = 2$ , an arrow is a pair like $(0.7, - 0.3)$ and we can literally draw it.

WHY these two, not one? Attention is a matching operation: a question meeting an answer. You cannot match one thing against nothing — you need two arrows to compare.

PICTURE. Two arrows from the same origin. The only thing that matters for matching is: do they point the same way?

Step 2 — The dot product: turning two arrows into one number

WHAT. To measure "do these arrows agree?" we multiply them component by component and add up the results. This single number is the dot product, written $Q K^{T}$ (the little $T$ just means we line them up to multiply — ignore it as machinery for now): $Q K^{T} = 1st pair q_{1} k_{1} + 2nd pair q_{2} k_{2} + \dots + last pair q_{d_{k}} k_{d_{k}} = \sum_{i = 1}^{d_{k}} q_{i} k_{i}$

Reading the equation left to right: take the first number of $Q$ times the first number of $K$ , then the second times the second, and so on for all $d_{k}$ pairs, then add every product into one total. The big $\sum$ symbol is just shorthand for "add all of these up, with $i$ counting from $1$ to $d_{k}$ ."

WHY the dot product and not, say, the distance between arrows? Because the dot product is large and positive when arrows point the same way, near zero when they are perpendicular, and negative when they point apart. That is exactly the "how much do they agree?" score attention wants. Distance would only tell us how far apart the tips are, which mixes up length with direction.

PICTURE. Same two arrows, now shown as they swing from aligned → perpendicular → opposed, with the resulting number dropping from big-positive to zero to negative.

Recall

Why does a dot product answer "do these agree?" and not "how far apart are they?" ::: Because it grows with alignment (direction), peaking when arrows point the same way and going negative when opposed — length matters less than direction here.

Step 3 — The score is a sum of random pieces

WHAT. When the network starts training, the numbers inside $Q$ and $K$ are basically random. A standard way to model "random with typical size 1, centred on 0" is to say each entry is drawn from $N (0, 1)$ — read as "a bell-shaped spread of values, centred at $0$ , whose typical wobble is $1$ ." So each product $q_{i} k_{i}$ is a small random number, and our score is the sum of $d_{k}$ of them: $Q K^{T} = q_{1} k_{1} + q_{2} k_{2} + \dots + q_{d_{k}} k_{d_{k}}$

WHY care that it is a sum? Here is the crux of the whole page: when you add up many independent random pieces, the total does not stay small — it spreads out more and more as you add more pieces. More pieces $\Rightarrow$ wilder total. And $d_{k}$ (the number of pieces) is often $64$ or more. So a big $d_{k}$ makes the score swing to huge magnitudes for no meaningful reason — purely from stacking randomness.

PICTURE. Three histograms of the score $Q K^{T}$ for $d_{k} = 1$ , $d_{k} = 8$ , $d_{k} = 64$ : same shape, but the bell gets visibly wider as $d_{k}$ grows.

Step 4 — Measuring the swing: variance, one tool for one job

WHAT. We need a number for "how wide is the spread of the score?" That number is the variance — the average of the squared distance from the centre. We write $Var (\cdot)$ . Two facts about variance are all we need, and both are pictures:

Independent things add their variances. If you stack $d_{k}$ independent random pieces, the total variance is the sum of their variances. $Var (\sum_{i = 1}^{d_{k}} q_{i} k_{i}) = \sum_{i = 1}^{d_{k}} Var (q_{i} k_{i})$ Each term on the right is the spread of one product; adding them gives the spread of the whole sum.
One product $q_{i} k_{i}$ has variance $1$ . Since $q_{i}$ and $k_{i}$ are each independent $N (0, 1)$ (spread $1$ , centre $0$ ), their product has variance $Var (q_{i} k_{i}) = Var (q_{i}) \cdot Var (k_{i}) = 1 \cdot 1 = 1$ .

Put them together: $Var (Q K^{T}) = \sum_{i = 1}^{d_{k}} 1 = d_{k}$ Reading it: each of the $d_{k}$ products contributes a spread of $1$ ; add $d_{k}$ ones; you get $d_{k}$ .

WHY variance and not just "the biggest value"? Because variance adds cleanly over independent pieces (fact 1). The maximum, the range, the average magnitude — none of them add so simply. Variance is the one measure of spread that turns "sum of pieces" into "sum of numbers", which is exactly the algebra we need.

PICTURE. A bar chart: $d_{k}$ little bars each of height $1$ (one per product), stacking into a total tower of height $d_{k}$ — the variance of the score literally is the count of pieces.

Recall

Why is the variance of the score exactly $d_{k}$ ? ::: It is a sum of $d_{k}$ independent products, each of variance $1$ ; independent variances add, so $1 + 1 + \dots + 1 = d_{k}$ .

Step 5 — What softmax does, and why big scores break it

WHAT. The scores feed into softmax, the function that turns a list of raw scores into a list of probabilities (positive numbers that sum to $1$ ). For scores $s_{1}, \dots, s_{n}$ : $softmax (s_{j}) = \frac{e ^{s_{j}}}{e ^{s_{1}} + e ^{s_{2}} + \dots + e ^{s_{n}}}$ The $e^{s}$ (exponential) blows small differences up: because $e$ grows so fast, a score even a little bigger than the rest ends up with almost all the probability.

WHY does big variance hurt? If the scores swing to $\pm$ huge (Step 4), then one $e^{s_{j}}$ dwarfs every other, so softmax outputs something like $(0, 0, 1, 0, \dots)$ — a spike. A spike means the model has committed 100% to one word before it has learned anything. Worse, at a spike the softmax is flat (its slope $\approx 0$ ), so the gradient — the signal that tells the network how to improve — vanishes. Learning stalls. This is the "saturation" the parent note warned about.

PICTURE. Two softmax outputs on the same raw pattern of scores: gentle scores → soft, spread-out weights (healthy); scores multiplied up by $d_{k}$ → a single spike (dead gradient).

Step 6 — The fix falls out: divide by $d_{k}$

WHAT. We know the score has variance $d_{k}$ (Step 4). We want variance back to $1$ (a gentle, un-saturated spread, Step 5). There is a rule for scaling: dividing a random quantity by a number $c$ divides its variance by $c^{2}$ . $Var (\frac{Q K ^{T}}{c}) = \frac{Var ( Q K ^{T} )}{c ^{2}} = \frac{d _{k}}{c ^{2}}$ Set this equal to $1$ and solve for $c$ : $\frac{d _{k}}{c ^{2}} = 1 ⟹ c^{2} = d_{k} ⟹ c = d_{k}$

That is the whole reason the paper writes $\frac{Q K ^{T}}{d _{k}}$ : it is the unique number that pulls the score's spread back down to $1$ , whatever $d_{k}$ is.

WHY $d_{k}$ and not $d_{k}$ ? A common wrong guess. Dividing by $d_{k}$ would give variance $d_{k} / d_{k}^{2} = 1/ d_{k}$ — now the scores are too flat, softmax gives everything equal weight, and attention learns nothing. We want variance exactly $1$ , and only $c = d_{k}$ achieves that.

PICTURE. The wide $d_{k} = 64$ histogram from Step 3, and next to it the same scores after dividing by $64 = 8$ — snapped back to the tidy unit-width bell.

Recall

Why $d_{k}$ rather than $d_{k}$ ? ::: Dividing by $c$ divides variance by $c^{2}$ ; to turn $d_{k}$ into $1$ we need $c^{2} = d_{k}$ , i.e. $c = d_{k}$ . Dividing by $d_{k}$ over-shrinks to $1/ d_{k}$ .

Step 7 — The degenerate cases (never leave a gap)

Good reproduction (per the parent's checklist) means checking the corners. Each is a quick sanity picture:

$d_{k} = 1$ . One number in each arrow, score variance $= 1$ already, and $1 = 1$ : the scaling divides by $1$ and does nothing. Correct — there is no stacked randomness to tame.
Scores all equal (perfectly tied words). Softmax gives a flat $(1/ n, \dots, 1/ n)$ : attention spreads evenly. The $d_{k}$ never changes this — scaling a constant leaves it constant. No blow-up possible.
One score $\to + \infty$ . Softmax $\to$ a one-hot spike; gradient $\to 0$ . This is the failure mode; scaling reduces how often we land here but cannot rescue an already-saturated score. That is why the parent's mask uses $- 1 0^{9}$ deliberately — a chosen extreme, not an accidental one.
$V$ constant across words. Output equals that constant no matter the weights: attention can only mix the values it is given.

The one-picture summary

Everything on this page in a single diagram: two random arrows → dot product → wide score → divide by $d_{k}$ → tame score → soft, learnable attention.

Recall Feynman retelling — say it to a friend

Every word becomes an arrow. To ask "how much should this word care about that one," we slide the two arrows together and add up their matching parts — that is the dot product, one number that is big when they point the same way. But each arrow holds $d_{k}$ numbers, and adding $d_{k}$ random little products makes the total swing wildly — the more numbers, the wilder, and the size of that swing (its variance) works out to exactly $d_{k}$ . A wild score is poison to softmax: one huge score grabs all the attention and the learning signal dies. So we shrink the score by dividing by $d_{k}$ — because dividing by a number cuts the swing by that number squared, and $d_{k}$ squared is $d_{k}$ , which cancels the swing perfectly back to size $1$ . Not $d_{k}$ (too much) — $d_{k}$ , exactly. That one square root is the whole trick, and now you could rederive it on a napkin.

See also: Attention Mechanisms for where this line lives in the full model, Statistical Significance Testing for judging whether a claimed gain is real, Experiment Tracking and Versioning and Hyperparameter Tuning for the reproduction run, Model Debugging for the overfit-10-samples sanity check, and Transfer Learning / Paper Writing and Publication for what comes after you have reproduced it.