Pretraining & Fine-Tuning LLMs

Q1 (14)

(a) LoRA params $= B + A$ sizes $= 4096\times16 + 16\times4096 = 65536 + 65536 = 131072$. (2) Full matrix $= 4096^2 = 16{,}777{,}216$. Percentage $= 131072 / 16777216 = 0.0078125 = 0.78125\%$. (2)

(b) Per matrix $131072$; matrices $= 32 \times 4 = 128$. Total $= 131072 \times 128 = 16{,}777{,}216 \approx 1.678\times10^7$. (2) Fraction of 7B $= 1.6777\times10^7 / 7\times10^9 = 0.002397 \approx 0.24\%$. (2)

(c) FP16: 7e9 × 2 bytes $= 14\times10^9 = 14$ GB. (1.5) NF4 (4-bit = 0.5 byte): 7e9 × 0.5 $= 3.5\times10^9 = 3.5$ GB. (1.5)

(d) At step 0, $BA = 0$ (since $B=0$), so $W' = W$ (base unchanged). (1.5) Chosen so training starts as an identity/no-op perturbation, preventing a random shock to the pretrained model and giving stable initial gradients. (1.5)

Q2 (12)

(a) Full FT of 13B updates all weights → must store a full 13B-parameter copy per task (20 copies ≈ huge storage). (2) Also with only ~2000 examples per task, full FT overfits and shifts weights far from pretrained values, causing catastrophic forgetting of general capabilities. (2)

(b) Use LoRA / adapters (PEFT): freeze the single shared base, train small per-task adapter weights (a few MB each). (2) At inference, keep one base in memory and hot-swap/merge the relevant task's small adapter for each request — 20 adapters cost negligible memory vs 20 full models. (2)

(c) Outperforms when the pretrained representations already linearly separate the task and data is tiny — few trainable params → less overfitting, faster, no forgetting. (2) Underperforms when the task needs the internal features to adapt (domain shift, complex mapping) — a frozen backbone can't learn new representations that PEFT/LoRA can inject. (2)

Q3 (12)

(a) Masked Language Modeling (MLM). (1) Randomly select ~15% of tokens; of those, 80% replaced with [MASK], 10% replaced with a random token, 10% left unchanged; model predicts the original tokens using full bidirectional context. (2) Not 100% masking because at fine-tuning/inference [MASK] never appears — the 80/10/10 split reduces the pretrain–finetune mismatch and forces the model to build robust representations of all tokens, not just masked positions. (2)

(b) Causal / autoregressive Language Modeling (next-token prediction). (1) Bidirectional attention would leak future tokens into the prediction of the current token, trivially letting the model "see the answer," so a causal mask restricting attention to left context is required. (2)

(c) Example (sentinels <X>, <Y>): Input: the <X> fox <Y> Target: <X> quick brown <Y> jumps <Z> (4) (Full marks if corrupted spans replaced by sentinels in input and target lists sentinel + original span content, terminated by a final sentinel.)

Q4 (12)

(a) INT8: 7e9 × 1 byte $= 7$ GB. (1.5) INT4: 7e9 × 0.5 byte $= 3.5$ GB. (1.5) Reduction of INT4 vs FP16 $= (14-3.5)/14 = 0.75 = 75\%$. (1)

(b) GPTQ quantizes weights column/group-wise while using a Hessian-based (second-order) approximation to measure the output error each rounding causes, then updates the not-yet-quantized weights to compensate for the error already introduced. Naive round-to-nearest treats each weight independently and lets errors accumulate; GPTQ's compensation keeps the layer's output close to the original, preserving accuracy at 4-bit. (4)

(c) $\mathcal{L} = \alpha\, T^2\, \mathrm{KL}(\sigma(z_t/T)\,\|\,\sigma(z_s/T)) + (1-\alpha)\,\mathrm{CE}(y, \sigma(z_s))$ (2) where $z_t,z_s$ = teacher/student logits, $\sigma$ = softmax, $y$ = ground-truth label, $\alpha$ = weighting, $T$ = temperature. (1) $T$ softens the distributions to expose the teacher's "dark knowledge" (relative probabilities of wrong classes); the $T^2$ factor rescales the KL gradient. (1)

Q5 (10)

(a) SFT on a narrow task: input–output pairs for one task, goal = maximize performance on that task. (1.5) Instruction tuning: diverse (instruction, response) pairs across many tasks phrased in natural language, goal = generalize to unseen instructions/tasks (zero-shot following). (1.5)

(b) Mechanism: gradient updates on the new distribution overwrite weights encoding previously learned general skills (no rehearsal of old data). (1) Mitigations (any two, 1.5 each): (i) PEFT/LoRA — freeze base, limit weight drift so original knowledge is retained; (ii) rehearsal / data mixing — include general-domain data in the SFT mix to keep those gradients present; (iii) low learning rate / early stopping / regularization toward base weights (e.g., EWC) — penalize moving important parameters. (3)

(c) Adding a KL/distillation term to the original base model's output distribution anchors the tuned model's predictions to the base behavior on general inputs, penalizing drift away from pretrained knowledge while still allowing the SFT loss to teach the new task — effectively a functional regularizer against forgetting. (3)

[
 {"claim":"LoRA params on 4096x4096 with r=16 = 131072 and 0.78125%","code":"p=4096*16+16*4096; full=4096**2; result=(p==131072 and abs(p/full-0.0078125)<1e-9)"},
 {"claim":"Total LoRA params over 128 matrices = 16777216 and ~0.24% of 7B","code":"tot=131072*128; frac=tot/7e9; result=(tot==16777216 and abs(frac-0.002396)<1e-4)"},
 {"claim":"FP16 14GB vs NF4 3.5GB for 7B; INT4 75% reduction vs FP16","code":"fp16=7e9*2/1e9; nf4=7e9*0.5/1e9; red=(14-3.5)/14; result=(fp16==14 and nf4==3.5 and abs(red-0.75)<1e-9)"},
 {"claim":"INT8 7GB, INT4 3.5GB for 7B","code":"i8=7e9*1/1e9; i4=7e9*0.5/1e9; result=(i8==7 and i4==3.5)"}
]