Level 1 — RecognitionScaling & Efficient Architectures

Scaling & Efficient Architectures

⏱ 20 minutes40 marksprintable — key stays hidden on paper

Level 1 Examination — Recognition

Time Limit: 20 minutes Total Marks: 40

Section A — Multiple Choice Questions (1 mark each)

Select the single best answer.

Q1. According to the Chinchilla scaling law, for compute-optimal training, model parameters $N$ and training tokens $D$ should be scaled: (a) $N$ fixed, $D$ increased only (b) Roughly equally (both proportional to $\sqrt{C}$ ) (c) $D$ fixed, $N$ increased only (d) $N$ proportional to $C$ , $D$ constant

Q2. In a Mixture-of-Experts (MoE) layer with top- $k$ routing, the number of expert FFNs activated per token is: (a) all experts (b) exactly $k$ (c) exactly 1 always (d) proportional to sequence length

Q3. The primary purpose of a load-balancing (auxiliary) loss in MoE is to: (a) reduce total parameter count (b) prevent all tokens routing to a few experts (c) increase the number of experts (d) speed up the softmax

Q4. ZeRO (Zero Redundancy Optimizer) reduces memory by: (a) reducing model depth (b) partitioning optimizer states, gradients, and parameters across devices (c) quantizing weights to int8 (d) removing the KV-cache

Q5. Tensor parallelism splits computation by: (a) assigning different layers to different devices (b) splitting individual weight matrices/operations across devices (c) replicating the full model on each device (d) sharding the dataset

Q6. Pipeline parallelism primarily suffers from which inefficiency? (a) load imbalance across experts (b) the "bubble" of idle time between stages (c) KV-cache overflow (d) softmax instability

Q7. State-space models such as Mamba/S4 offer what asymptotic advantage over standard attention for sequence length $L$ ? (a) $O(L^2)$ compute like attention (b) $O(L)$ / linear scaling in sequence length (c) $O(L^3)$ compute (d) constant memory regardless of parameters

Q8. Speculative decoding accelerates inference by: (a) skipping the softmax (b) using a small draft model to propose tokens that a large model verifies in parallel (c) reducing model parameters at runtime (d) caching all previous prompts permanently

Q9. The KV-cache stores which quantities during autoregressive generation? (a) query vectors only (b) key and value vectors of past tokens (c) gradients of past tokens (d) attention output logits

Q10. "Emergent abilities" in large models refer to capabilities that: (a) appear gradually and linearly with scale (b) appear abruptly beyond a certain scale threshold and are absent in smaller models (c) only exist in MoE models (d) are guaranteed by the scaling laws to be smooth

Section B — Matching (1 mark each, 6 marks)

Q11. Match each technique (A–F) to its correct description (1–6).

Technique		Description
A. FSDP		1. Small model proposes, large model verifies tokens
B. Gating network		2. Shards parameters, grads, optimizer states; reshards after use
C. Speculative decoding		3. Assigns tokens to experts via learned routing scores
D. Grouped-query attention		4. Compute-optimal ratio of params to tokens
E. Chinchilla		5. Reduces KV-cache by sharing K/V heads across queries
F. Pipeline parallelism		6. Splits model into sequential stages across devices

Section C — True/False WITH Justification (3 marks each: 1 T/F + 2 justification)

Q12. Increasing model parameters while holding training tokens fixed always improves loss without limit. (True/False + justify)

Q13. In sparse MoE, total parameter count can grow while the FLOPs per token stay roughly constant. (True/False + justify)

Q14. Data parallelism alone (with full replication) reduces the per-device memory required to store model parameters. (True/False + justify)

Q15. The KV-cache grows linearly with the generated sequence length. (True/False + justify)

Q16. Speculative decoding changes the output distribution of the large (target) model. (True/False + justify)

Q17. Emergent abilities can always be reliably extrapolated from small-model performance using smooth scaling curves. (True/False + justify)

Q18. Load-balancing loss is unnecessary because the gating softmax naturally distributes tokens evenly. (True/False + justify)

End of Paper

Answer keyMark scheme & solutions

Section A (10 × 1 = 10 marks)

Q1 — (b). Chinchilla found optimal scaling with $N \propto C^{0.5}$ and $D \propto C^{0.5}$ , i.e. parameters and data scale roughly equally. Prior (Kaplan) work over-weighted $N$ ; Chinchilla corrected this. (1)

Q2 — (b). Top- $k$ routing activates exactly $k$ experts per token — the defining sparse-activation property. (1)

Q3 — (b). Without balancing, routing collapses to a few "popular" experts; the auxiliary loss encourages uniform utilization. (1)

Q4 — (b). ZeRO stages partition optimizer states (Z1), gradients (Z2), and parameters (Z3) across data-parallel ranks, eliminating redundant copies. (1)

Q5 — (b). Tensor parallelism splits within an operation (e.g., columns/rows of weight matrices) — distinct from pipeline (layer-wise) parallelism. (1)

Q6 — (b). The pipeline "bubble" is idle time while stages wait for activations; micro-batching reduces but doesn't eliminate it. (1)

Q7 — (b). SSMs scale linearly $O(L)$ in sequence length vs attention's $O(L^2)$ , enabling long contexts. (1)

Q8 — (b). A cheap draft model proposes several tokens; the target model verifies them in one parallel forward pass, accepting a prefix. (1)

Q9 — (b). Keys and values of previous tokens are cached to avoid recomputation; queries are recomputed each step. (1)

Q10 — (b). Emergence = abrupt appearance of capability past a scale threshold, not present in smaller models. (1)

Section B (6 × 1 = 6 marks)

Q	Match	Reason
A. FSDP	2	Fully-Sharded Data Parallel shards & reshards params/grads/optim states
B. Gating network	3	Produces routing scores to select experts
C. Speculative decoding	1	Draft proposes, target verifies
D. Grouped-query attention	5	Shares K/V across query heads → smaller KV-cache
E. Chinchilla	4	Compute-optimal param:token ratio
F. Pipeline parallelism	6	Sequential stages across devices

Q11: A-2, B-3, C-1, D-5, E-4, F-6. (1 each)

Section C (7 × 3 = 21 marks)

Q12 — FALSE. (1) Justification: With fixed data, over-parameterized models eventually overfit / hit diminishing returns; Chinchilla shows loss depends on both $N$ and $D$ — you must scale tokens too. Loss has an irreducible floor. (2)

Q13 — TRUE. (1) Justification: MoE activates only top- $k$ of many experts per token, so FLOPs/token depend on active experts, not total. Adding experts raises parameter count while active compute stays fixed. (2)

Q14 — FALSE. (1) Justification: Pure data parallelism replicates the full model on every device, so per-device parameter memory is unchanged. Only ZeRO/FSDP sharding reduces it. (2)

Q15 — TRUE. (1) Justification: Cache stores K,V per past token per layer; cache size $\propto$ (sequence length) $\times$ layers $\times$ heads $\times$ head-dim — linear in length. (2)

Q16 — FALSE. (1) Justification: Speculative decoding uses a verification/rejection scheme that provably preserves the target model's exact output distribution; it only speeds up sampling. (2)

Q17 — FALSE. (1) Justification: Emergent abilities appear abruptly and are (by definition) hard to predict from small-model smooth curves — sometimes near-random until threshold. (Note: some argue metric choice explains this, but reliable extrapolation is not guaranteed.) (2)

Q18 — FALSE. (1) Justification: Softmax gating tends to collapse onto a few experts (rich-get-richer), so an explicit auxiliary balancing loss is needed for even utilization. (2)

[
  {"claim": "Chinchilla compute-optimal: with C=6*N*D, and N,D both proportional to sqrt(C), the product N*D scales as C.", "code": "C=symbols('C',positive=True); N=sqrt(C); D=sqrt(C); result = simplify(N*D - C)==0"},
  {"claim": "MoE active FLOPs fraction with 8 experts top-2 is 2/8 = 0.25 of dense-equivalent expert compute.", "code": "result = Rational(2,8)==Rational(1,4)"},
  {"claim": "KV-cache size scales linearly: doubling sequence length L from 1000 to 2000 doubles cache entries (layers,heads,dim fixed).", "code": "L1=1000; L2=2000; layers=32; heads=32; dim=128; cache=lambda L: L*layers*heads*dim*2; result = cache(L2)==2*cache(L1)"},
  {"claim": "Attention cost O(L^2) exceeds SSM O(L) for L>1: ratio L^2/L = L grows.", "code": "L=symbols('L',positive=True); ratio=(L**2)/L; result = simplify(ratio - L)==0"}
]