Level 2 — RecallPretraining & Fine-Tuning LLMs

Pretraining & Fine-Tuning LLMs

⏱ 30 minutes50 marksprintable — key stays hidden on paper

Time limit: 30 minutes
Total marks: 50
Instructions: Answer all questions. Show working where numerical.

Q1. (4 marks) Define self-supervised pretraining. Name and briefly describe the two main pretraining objectives used by (a) GPT-style and (b) BERT-style models.

Q2. (5 marks) Contrast the architectures of GPT, BERT, and T5 in terms of encoder/decoder structure and attention masking. Give one sentence per model.

Q3. (4 marks) Explain the text-to-text framework used by T5. Give two example tasks and how they are cast as text-to-text problems.

Q4. (6 marks) LoRA numerical. A weight matrix $W \in \mathbb{R}^{d \times k}$ with $d = 4096$ , $k = 4096$ is fine-tuned. LoRA replaces the update $\Delta W$ with a low-rank product $BA$ where $B \in \mathbb{R}^{d \times r}$ , $A \in \mathbb{R}^{r \times k}$ , and rank $r = 8$ . (a) How many parameters does full fine-tuning of $W$ update? (2) (b) How many trainable parameters does LoRA introduce for this layer? (2) (c) What fraction (as a percentage, 3 s.f.) of the full-tuning count is this? (2)

Q5. (5 marks) Distinguish full fine-tuning from feature extraction (freezing the backbone). State one advantage and one disadvantage of each.

Q6. (5 marks) Define catastrophic forgetting. Name two techniques used to mitigate it during fine-tuning.

Q7. (6 marks) Quantization numerical. A model has $7 \times 10^9$ parameters stored in FP16 (16 bits each). (a) Compute the memory in GB (use $1\text{ GB} = 10^9$ bytes). (2) (b) Compute the memory if quantized to INT4. (2) (c) State the compression ratio FP16:INT4. (2)

Q8. (5 marks) Compare adapter layers and prefix tuning as PEFT methods. For each, state where the trainable parameters are inserted.

Q9. (5 marks) Define knowledge distillation. Explain the roles of the teacher and student, and state what "soft labels" (soft targets) are.

Q10. (5 marks) Define instruction tuning and supervised fine-tuning (SFT). State one way they differ in the data used.

Answer keyMark scheme & solutions

Q1. (4 marks)

Self-supervised pretraining: training on unlabelled text where supervision signal is derived from the data itself (e.g. predicting hidden/next tokens). (2)
(a) GPT: causal / autoregressive language modelling — predict next token given previous tokens. (1)
(b) BERT: masked language modelling (MLM) — predict randomly masked tokens using bidirectional context (often + next-sentence prediction). (1)

Q2. (5 marks)

GPT: decoder-only, uses causal (unidirectional) masked self-attention. (2)
BERT: encoder-only, uses full bidirectional self-attention (no causal mask). (2)
T5: encoder–decoder; encoder bidirectional, decoder causal with cross-attention. (1)

Q3. (4 marks)

Every NLP task is framed as mapping input text → output text with a task prefix. (2)
Examples (any 2, 1 each): translation "translate English to German: ..." → German text; summarization "summarize: ..." → summary; classification "sst2 sentence: ..." → "positive"/"negative". (2)

Q4. (6 marks)

(a) Full: $d \times k = 4096 \times 4096 = 16{,}777{,}216$ parameters. (2)
(b) LoRA: $B$ has $d\cdot r = 4096\times 8 = 32768$ ; $A$ has $r\cdot k = 8\times4096 = 32768$ ; total $= 65{,}536$ . (2)
(c) Fraction $= 65536 / 16777216 = 0.00390625 = 0.391\%$ (3 s.f.). (2)

Q5. (5 marks)

Full fine-tuning: update all model weights. (1) Adv: highest task performance / flexibility. (1) Disadv: costly memory/compute, risk of forgetting. (0.5)
Feature extraction: freeze backbone, train only new head. (1) Adv: cheap, fast, low overfitting risk. (1) Disadv: lower ceiling, cannot adapt internal representations. (0.5)

Q6. (5 marks)

Catastrophic forgetting: when learning a new task, the model overwrites/loses previously learned knowledge, degrading prior-task performance. (3)
Mitigations (any 2, 1 each): rehearsal/replay of old data, regularization (e.g. EWC), lower learning rate, PEFT/LoRA (freeze base), multi-task training. (2)

Q7. (6 marks)

(a) FP16: $7\times10^9 \times 16$ bits $= 7\times10^9 \times 2$ bytes $= 1.4\times10^{10}$ bytes $= 14$ GB. (2)
(b) INT4: $7\times10^9 \times 4$ bits $= 7\times10^9 \times 0.5$ bytes $= 3.5\times10^9$ bytes $= 3.5$ GB. (2)
(c) Ratio $= 16:4 = 4:1$ (14/3.5 = 4). (2)

Q8. (5 marks)

Adapter layers: small bottleneck feed-forward modules inserted inside each transformer block (after attention/FFN); only adapters trained. (2.5)
Prefix tuning: trainable continuous prefix vectors prepended to the keys/values (or input) at each attention layer; base model frozen. (2.5)

Q9. (5 marks)

Knowledge distillation: transferring knowledge from a large model to a smaller one. (1)
Teacher: large pretrained model producing target outputs. (1) Student: smaller model trained to mimic teacher. (1)
Soft labels: the teacher's full output probability distribution (often temperature-scaled), richer than hard one-hot labels. (2)

Q10. (5 marks)

SFT: supervised fine-tuning on labelled input–output pairs to adapt behaviour. (2)
Instruction tuning: fine-tuning on datasets of (instruction, response) across many tasks so the model follows natural-language instructions/generalises to unseen tasks. (2)
Difference: instruction tuning uses diverse instruction-formatted multi-task data; plain SFT may target a single/narrow task. (1)

[
  {"claim":"Full FT params = 4096*4096 = 16777216","code":"result = (4096*4096 == 16777216)"},
  {"claim":"LoRA params = 2*4096*8 = 65536","code":"result = (2*4096*8 == 65536)"},
  {"claim":"LoRA fraction = 0.00390625","code":"result = (Rational(65536,16777216) == Rational(1,256))"},
  {"claim":"FP16 mem 14GB, INT4 3.5GB, ratio 4","code":"fp16=7e9*2; int4=7e9*0.5; result = (abs(fp16-14e9)<1 and abs(int4-3.5e9)<1 and abs(fp16/int4-4)<1e-9)"}
]