4.3.1 · D5Pretraining & Fine-Tuning LLMs
Question bank — GPT family architecture evolution
This bank drills the conceptual skeleton — masking, decoder-only design, pre-LN, in-context learning, scaling, RLHF, MoE. Nothing here needs a calculator; the heavy arithmetic lives in the D3/D4 pages.
True or false — justify
GPT uses the same encoder–decoder structure as the 2017 Transformer paper.
False. GPT is decoder-only — it keeps only the masked self-attention stack, with no encoder and no cross-attention, because language modeling has no separate input to encode. See BERT vs GPT (encoder vs decoder).
The chain-rule factorization is an approximation GPT makes.
False. It is exact for any joint distribution; it follows from repeatedly applying . GPT's approximation is only in the neural net that models each factor.
The causal mask is applied after the softmax.
False. The mask adds to future scores before softmax, so those weights become exactly after softmax. Masking after softmax would leave nonzero leakage that no renormalization fully removes.
GPT-3 was fine-tuned separately on each downstream task.
False. GPT-3's headline result is few-shot in-context learning — examples go in the prompt with frozen weights. Only GPT-1 did per-task supervised fine-tuning.
Pre-LN and post-LN produce identical training dynamics.
False. Post-LN deep stacks suffer exploding/vanishing residual signal; pre-LN keeps a clean residual highway, which is what let GPT-2/3 train 48–96 layers stably.
Scaling laws predict that test loss can be driven to zero with enough compute.
False. A power law decreases but toward an irreducible floor; each 10× gives only a fixed fractional drop, never a jump to zero. See Scaling laws (Kaplan, Chinchilla).
RLHF changed GPT's architecture.
False. RLHF (GPT-3.5 / InstructGPT) is an alignment / training-procedure change on the same decoder backbone — the network layers are unchanged. See RLHF and InstructGPT.
The divisor is an arbitrary normalization convention.
False. It specifically cancels the variance growth of the dot product (), keeping scores so softmax stays in a region with usable gradients.
Every GPT version introduced a fundamentally new attention mechanism.
False. The core is the same causal self-attention; only GPT-3 added alternating dense/locally-banded sparse attention, and GPT-4 is believed to use MoE routing — the attention math itself barely changed.
Spot the error
"Because GPT is generative, it must attend to future tokens to plan ahead."
Error: generation is left-to-right; a token predicts the next using only past context. Attending to the future would let it see the answer during training — the model would cheat and learn nothing.
"Widening the model from to doubles the attention parameters."
Error: attention params scale as ==== (four projections), so doubling quadruples them, not doubles. This quadratic growth is why width is so parameter-hungry.
"Since GPT-4 is bigger, it proves architecture progress, not scale."
Error: the scaling philosophy is unchanged; GPT-4's additions (MoE, multimodality) sit on the same decoder skeleton. The through-line remains "scale next-token prediction". See Mixture-of-Experts.
"The log in the loss is just for numerical prettiness."
Error: the turns a product of tiny probabilities (which would underflow) into a sum, and yields well-behaved gradients — it is functionally essential, not cosmetic. See Cross-entropy loss.
"Zero-shot means GPT-2 needs zero examples because it memorized every task."
Error: zero-shot means it transfers from language-modeling pretraining to unseen tasks without task-specific weights or examples — not memorization of tasks it never trained on.
"Residual connections are there to add more parameters."
Error: residuals add no parameters; they create an identity path so gradients flow through deep stacks without vanishing.
Why questions
Why decoder-only instead of keeping the encoder?
Pure language modeling has no separate input sequence to encode and never needs future context, so the encoder and cross-attention are dead weight — one masked-decoder tower is simpler and scales cleaner.
Why does the scaling law show as a straight line on a log-log plot?
A straight line on log-log axes is the signature of a power law ; each fixed multiplicative increase in gives a fixed additive drop in .
Why did OpenAI dare build 175B parameters before seeing it work?
Because scaling laws made loss predictable — they could forecast the loss drop from the curve fit on smaller models, de-risking the huge investment.
Why is the causal mask "the thing" that makes GPT autoregressive?
It enforces that position can only see positions , so factorizing is architecturally guaranteed rather than just hoped for. See Self-attention and multi-head attention.
Why move LayerNorm inside the residual branch (pre-LN)?
It leaves the residual stream un-normalized as a clean gradient highway, preventing the residual signal from exploding/vanishing across very deep stacks. See Transformer architecture.
Why does in-context learning "emerge" only at GPT-3 scale?
It is an emergent property: below a scale threshold the model lacks the capacity to infer task patterns from a few prompt examples; enough parameters/data unlock it without any new training objective.
Edge cases
What does the causal mask do for the very first token (position 1)?
It can attend only to itself — the entire future is masked — so its prediction relies purely on its own embedding plus positional information. See Positional encodings.
What happens to attention scores if is very large and we don't divide by ?
Scores grow with , pushing softmax into a saturated, near-one-hot regime where gradients vanish and learning stalls.
Is fine-tuning ever still used in the GPT family, or fully replaced by prompting?
Fine-tuning is optional, not obsolete — GPT-3 made it unnecessary for many tasks via few-shot prompting, but task-specific fine-tuning and RLHF still exist for alignment and specialization.
If two GPT versions have identical parameter counts but one trains on far more tokens, which wins?
Generally the data-richer one, by scaling laws loss depends on both and ; Chinchilla showed many models were under-trained on data relative to their size. See Scaling laws (Kaplan, Chinchilla).
For a sequence of length , is the causal mask meaningful?
It is degenerate but valid — the single token attends only to itself, so the mask changes nothing, and the loss is just the negative log-probability of that one token given no context.