Intuition The One Core IdeaMasked attention is a way of letting each word in a sentence look back at the words before it — and forbidding it from looking forward — so the model learns to predict the next word instead of copying it. Everything on the parent page is just machinery for drawing an invisible curtain that hides the future from each position.
Before you can read the parent note, you must own every symbol it throws at you. Below, each symbol is built from nothing: plain words → the picture → why the topic needs it. They are ordered so each one leans only on the ones above it.
Definition Token and position indexA token is one chunk of text (a word or word-piece) that the model reads. We line the tokens up left to right and give each one a number called its position index , written i j
Picture a row of numbered boxes:
i doing the looking (the "query" position).j being looked at (the "key" position).T
Intuition Why two different letters?Attention always compares a pair: "how much should position i j two running numbers because we are filling in a whole grid of pairs ( i , j )
Definition A vector as a list of numbersA vector is just an ordered list of numbers, e.g. [ 1 , 2 ] arrow in space; a list of 2 numbers is an arrow on a flat page, a list of d k d k d k dimension — how many numbers are in each list.
Every token is turned into three such vectors. The parent note stacks them as rows of three matrices:
Symbol
Plain words
Picture
Q "What am I looking for?"
an arrow each position points out with
K "What do I offer?"
an arrow each position advertises
V "What do I hand over if chosen?"
the actual content passed along
Intuition Why three roles, not one?Think of a library search. Your search box is the Query , each book's spine label is a Key , and the book's contents are the Value . You match your query against labels, then collect the contents of the best matches. Attention is exactly this library lookup, done with arrows. See Self-Attention Mechanism for how Q , K , V
The matrix Q T × d k T d k K V
The dot product of two vectors multiplies matching entries and adds them up:
[ a 1 , a 2 ] ⋅ [ b 1 , b 2 ] = a 1 b 1 + a 2 b 2 how much do these two arrows point the same way? Big positive = aligned, near zero = perpendicular (unrelated), negative = opposite.
Intuition Why the dot product and not, say, subtraction?We want a single number that says "these two directions agree." Subtraction gives another vector, not a score. The dot product is the standard tool that squeezes two arrows into one alignment score — exactly what "how much should i j every query with every key we get the whole grid at once, and that grid is written Q K T
K T T not the sequence length) means transpose : flip a matrix so its rows become columns. We flip K Q every query-key dot product in one shot.
So A = Q K T T × T A ij i ⋅ j i j
Definition Square root and why we divided k d k d k Q K T
Intuition Why this exact tool?When you add up d k d k d k Attention Score Scaling for the full reason. Without it, scores blow up and gradients vanish.
Definition The exponentiale x e x any number into a positive number, and turns bigger inputs into much-bigger outputs (it grows explosively). Two facts we lean on:
e 0 = 1 e − ∞ = 0
Softmax turns a row of scores into a row of weights that are all positive and add up to 1 — a probability distribution:
softmax ( s ) j = ∑ k e s k e s j 1
Intuition Why softmax and not just "divide by the sum"?We need every weight to be positive (a negative "amount of attention" is meaningless) and we want big scores to dominate. Plain division fails on negative scores; the exponential fixes both — it forces positivity and sharpens the contrast between high and low scores. The symbol ∑
This is the hinge of the whole topic: because e − ∞ = 0 − ∞ before softmax, that position gets weight exactly 0 1
Definition Negative infinity as "impossible"− ∞ flag meaning "this score is unreachably small." After softmax it becomes weight 0 "forbidden — pretend this connection does not exist."
Definition The causal maskM M T × T M ij = { 0 − ∞ j ≤ i ( past or self — allowed ) j > i ( future — forbidden ) add M M = 0 M = − ∞
Definition Lower-triangular matrixA lower-triangular matrix keeps the entries on and below the main diagonal (top-left to bottom-right) and zeros above it. The allowed region of M 0 i 1 … i
j ≤ i j = i j = i itself — that is allowed and useful. We only forbid j > i future . (The parent note's "Mistake 3" is exactly this trap.)
α ij how much of position j i Each row of α
Definition The product symbol∏ ∏ ∑ P ( y ) = ∏ t = 1 T P ( y t ∣ y < t ) the probability of the whole sentence = multiply, for every position t t
Definition The "given" bar andy < t ∣ y < t y less than t no peeking ahead , and the mask is what enforces it inside the network. See Decoder Architecture and Teacher Forcing for where this factorization comes from.
Tokens and positions i j T
Neg infinity gives weight zero
Causal mask M lower triangular
Masked attention for autoregression
Every arrow feeds into the parent topic . Adjacent tools you will meet next: Positional Encoding , KV Caching , Encoder-Decoder Models .
Cover the right side and check you can answer each before moving on.
What does the index i j i j
What is a vector, in one phrase? An ordered list of numbers, pictured as an arrow in space.
What are Q K V Query = your search, Key = book label, Value = book contents.
What single question does a dot product answer? How much do these two arrows point the same way (their alignment)?
What does the superscript T K T Transpose — flips rows into columns so every query meets every key.
Why divide scores by d k To cancel the natural growth of dot-product size with dimension, keeping scores tame.
What two facts about e x e 0 = 1 e − ∞ = 0
What does softmax guarantee about a row of weights? All positive and they sum to exactly 1.
Why add − ∞ Because e − ∞ = 0
What shape does the allowed region of M Lower-triangular — position i 1 … i
Is the diagonal (j = i No — a position may attend to itself; only j > i
What does ∏ t P ( y t ∣ y < t ) The sentence probability is the product, over each position, of that word's probability given all earlier words.
