Level 1 — RecognitionDeep Learning Frameworks

Deep Learning Frameworks

⏱ 20 minutes30 marksprintable — key stays hidden on paper

Chapter: 3.3 Deep Learning Frameworks Level: 1 — Recognition (MCQ + Matching + True/False with justification) Time Limit: 20 minutes Total Marks: 30

Section A — Multiple Choice (1 mark each, 10 marks)

Choose the single best answer.

Q1. Which PyTorch attribute must be set to True for a tensor to track operations for gradient computation?

(a) requires_grad
(b) is_leaf
(c) retain_graph
(d) grad_fn

Q2. Calling loss.backward() in PyTorch does which of the following?

(a) Updates the model weights
(b) Computes gradients and stores them in .grad of leaf tensors
(c) Zeroes the gradients
(d) Moves tensors to the GPU

Q3. What is the correct order of the three core steps inside a standard PyTorch training loop iteration (after the forward pass produces a loss)?

(a) optimizer.step() → loss.backward() → optimizer.zero_grad()
(b) optimizer.zero_grad() → loss.backward() → optimizer.step()
(c) loss.backward() → optimizer.zero_grad() → optimizer.step()
(d) optimizer.zero_grad() → optimizer.step() → loss.backward()

Q4. In PyTorch, which method correctly moves a tensor x to the GPU?

(a) x.gpu()
(b) x.to('cuda')
(c) x.cuda_move()
(d) x.device('cuda')

Q5. Which class must you subclass to define a custom neural network in PyTorch?

(a) torch.Tensor
(b) torch.optim.Optimizer
(c) torch.nn.Module
(d) torch.utils.data.Dataset

Q6. The recommended way to save a model for later reuse in PyTorch is to save the:

(a) entire Python script
(b) model.state_dict()
(c) optimizer only
(d) computational graph

Q7. In a PyTorch DataLoader, the batch_size argument controls:

(a) the number of epochs
(b) the number of samples returned per iteration
(c) the learning rate
(d) the number of GPUs used

Q8. In Keras, which method starts the training process of a compiled model?

(a) model.train()
(b) model.fit()
(c) model.run()
(d) model.backward()

Q9. Mixed precision training primarily uses which numeric format alongside FP32 to speed up computation and save memory?

(a) INT8
(b) FP64
(c) FP16 (half precision)
(d) BF64

Q10. In distributed data-parallel training, the dataset is typically:

(a) copied entirely to every GPU each step
(b) split so each worker processes a different shard of the data
(c) processed only on the CPU
(d) never shuffled

Section B — Matching (1 mark each, 8 marks)

Match each term in Column X with its correct description in Column Y.

#	Column X		Column Y
Q11	`autograd`	A	Utility that batches, shuffles, and iterates over a `Dataset`
Q12	`DataLoader`	B	Scales the loss to prevent FP16 gradient underflow
Q13	`GradScaler`	C	Automatic differentiation engine building a computational graph
Q14	TensorBoard	D	Container/interface holding layers and learnable parameters
Q15	`nn.Module`	E	Visualization tool for logging scalars, graphs, and metrics
Q16	`optimizer.zero_grad()`	F	Clears accumulated gradients before backprop
Q17	`state_dict`	G	Dictionary mapping each layer to its learnable tensors
Q18	`.item()`	H	Extracts a Python scalar from a single-element tensor

Section C — True / False with Justification (2 marks each, 12 marks)

State True or False and give a one-line justification. (1 mark answer + 1 mark justification.)

Q19. In PyTorch, gradients accumulate by default, so you must zero them each iteration.

Q20. model.eval() disables gradient computation entirely, making torch.no_grad() unnecessary.

Q21. A tensor created directly by the user with requires_grad=True is a leaf tensor.

Q22. In mixed precision training, keeping master weights in FP32 helps maintain numerical stability during the optimizer update.

Q23. Weights & Biases (wandb) and TensorBoard are both tools used primarily for experiment tracking and logging.

Q24. Moving a model to GPU with model.to('cuda') automatically moves the input data batches to the GPU too.

Answer keyMark scheme & solutions

Section A (10 marks)

Q1 — (a) requires_grad. This flag tells autograd to track operations on the tensor. grad_fn and is_leaf are read-only properties, not switches. (1)

Q2 — (b). backward() triggers reverse-mode autodiff, populating .grad of leaf tensors. Weight update is optimizer.step(); zeroing is optimizer.zero_grad(). (1)

Q3 — (b) zero_grad → backward → step. Clear old gradients, compute new ones, then apply update. (1)

Q4 — (b) x.to('cuda'). The .to(device) API (or x.cuda()) is the correct transfer call. (1)

Q5 — (c) torch.nn.Module. Custom models inherit from nn.Module and implement forward(). (1)

Q6 — (b) model.state_dict(). Saving the state dict (parameter/buffer tensors) is portable and recommended over pickling the whole object. (1)

Q7 — (b). batch_size = samples per iteration/batch. (1)

Q8 — (b) model.fit(). Keras' high-level training entry point. (1)

Q9 — (c) FP16. Half precision speeds compute and halves memory; FP32 master copy retained. (1)

Q10 — (b). Data-parallel sharding: each worker gets a distinct data slice; gradients are synchronized. (1)

Section B (8 marks)

Q	Answer
Q11 `autograd`	C — automatic differentiation engine
Q12 `DataLoader`	A — batches/shuffles/iterates a Dataset
Q13 `GradScaler`	B — scales loss to avoid FP16 underflow
Q14 TensorBoard	E — visualization/logging tool
Q15 `nn.Module`	D — container holding layers/parameters
Q16 `zero_grad()`	F — clears accumulated gradients
Q17 `state_dict`	G — dict mapping layers to learnable tensors
Q18 `.item()`	H — extracts Python scalar from 1-element tensor

(1 mark each; award only for exact matches.)

Section C (12 marks)

Q19 — True. (1) PyTorch accumulates (+=) gradients into .grad across backward() calls; hence optimizer.zero_grad() is required each iteration to prevent mixing. (1)

Q20 — False. (1) model.eval() only switches modules like Dropout/BatchNorm to eval behaviour; it does not stop gradient tracking. Use torch.no_grad() to disable grad computation. (1)

Q21 — True. (1) User-created tensors with requires_grad=True (not produced by an operation) have no grad_fn and are leaves; their gradients are accumulated in .grad. (1)

Q22 — True. (1) FP32 master weights prevent precision loss when applying small updates that FP16 would round to zero, keeping training stable. (1)

Q23 — True. (1) Both are experiment tracking/logging tools for scalars, metrics, and visualizations; W&B additionally offers cloud dashboards. (1)

Q24 — False. (1) Moving the model does not move data. Each input batch must be explicitly transferred (e.g., x = x.to('cuda')); tensor device must match model device or a runtime error occurs. (1)

[
  {"claim":"Q7 batch_size semantics: total samples / batch_size = number of batches (1000/32 = 32 batches, ceil)","code":"import math; result = math.ceil(1000/32) == 32"},
  {"claim":"Q9 FP16 halves memory vs FP32 (16 bits vs 32 bits)","code":"result = (16/32) == 0.5"},
  {"claim":"Q19 gradient accumulation: sum of two grads of 3 equals 6 without zeroing","code":"g = 0; g += 3; g += 3; result = (g == 6)"},
  {"claim":"Q3 training step order index: zero_grad(0) < backward(1) < step(2)","code":"order = ['zero_grad','backward','step']; result = order.index('zero_grad') < order.index('backward') < order.index('step')"}
]