Level 2 — RecallDeep Learning Frameworks

Deep Learning Frameworks

⏱ 30 minutes40 marksprintable — key stays hidden on paper

Chapter: 3.3 Deep Learning Frameworks Difficulty Level: 2 — Recall (definitions, standard problems, short derivations) Time Limit: 30 minutes Total Marks: 40

Use $...$ for inline math where needed. Assume PyTorch/TensorFlow conventions.

Q1. Define a PyTorch tensor. State any two differences between a PyTorch tensor and a NumPy array. (4 marks)

Q2. A tensor is created with x = torch.tensor([2.0, 3.0], requires_grad=True) and $y = \sum x_i^2$ . (a) Write the expression for $y$ in terms of $x_1, x_2$ . (1 mark) (b) Derive $\dfrac{\partial y}{\partial x_1}$ and $\dfrac{\partial y}{\partial x_2}$ , then give the numeric gradient vector after y.backward(). (3 marks)

Q3. In the context of PyTorch's autograd, explain in one line each: (a) computational graph, (b) .backward(), (c) torch.no_grad(), (d) leaf tensor. (4 marks)

Q4. When building a custom model by subclassing nn.Module, name the two methods you must typically define and state the role of each. (4 marks)

Q5. Distinguish between a Dataset and a DataLoader in PyTorch. State two arguments commonly passed to a DataLoader. (4 marks)

Q6. Arrange the following steps of a from-scratch PyTorch training loop (for one batch) into the correct order, and state why optimizer.zero_grad() is required: (5 marks) loss.backward(), optimizer.step(), optimizer.zero_grad(), output = model(x), loss = criterion(output, y)

Q7. GPU / device management: (5 marks) (a) Write the one-line idiom to select GPU if available, else CPU. (1 mark) (b) Why must both the model and the input tensors be moved to the same device? (2 marks) (c) What does tensor.to(device) return — does it modify in place? (2 marks)

Q8. Checkpointing: (4 marks) (a) What object is conventionally saved to store a model's learned parameters? (1 mark) (b) Give the recommended two-line pattern (save then load) for that object. (2 marks) (c) Why is saving state_dict() preferred over saving the entire model object? (1 mark)

Q9. Keras basics: State the three-step high-level workflow to train a model with the Keras Sequential API (name the method used at each step). (3 marks)

Q10. Mixed precision & logging: (3 marks) (a) What is the purpose of a gradient scaler (e.g. torch.cuda.amp.GradScaler) in mixed precision training? (2 marks) (b) Name one tool used to visualize training curves (loss/accuracy over steps). (1 mark)

End of paper

Answer keyMark scheme & solutions

Q1. (4 marks)

Definition: A PyTorch tensor is a multi-dimensional array (n-dimensional) that supports GPU acceleration and automatic differentiation. (2)
Two differences (1 each):
- Tensors can run on GPU; NumPy arrays are CPU-only. (1)
- Tensors integrate with autograd (requires_grad) for automatic gradients; NumPy arrays do not. (1) (Accept: tensors support mixed precision, shared memory bridge .numpy().)

Q2. (4 marks) (a) $y = x_1^2 + x_2^2$ . (1) (b) $\dfrac{\partial y}{\partial x_i} = 2x_i$ — why: derivative of $x_i^2$ . (1) With $x_1=2, x_2=3$ : gradient $= [2\cdot2,\ 2\cdot3] = [4.0, 6.0]$ . (2)

Q3. (4 marks — 1 each) (a) Computational graph: a DAG recording operations on tensors used to compute gradients via backprop. (b) .backward(): triggers reverse-mode autodiff, populating .grad of leaf tensors. (c) torch.no_grad(): context manager that disables gradient tracking (used in inference/eval to save memory). (d) Leaf tensor: a tensor created directly by the user (not a result of an operation) with requires_grad=True; its .grad accumulates gradients.

Q4. (4 marks)

__init__: defines/registers the layers and submodules (parameters). (2)
forward: defines how input flows through layers to produce output (the forward computation). (2)

Q5. (4 marks)

Dataset: abstraction that stores/returns individual samples, implementing __len__ and __getitem__. (1.5)
DataLoader: wraps a Dataset to yield batches, handling shuffling, batching, and parallel loading. (1.5)
Two common args (0.5 each): batch_size, shuffle (accept num_workers, drop_last). (1)

Q6. (5 marks) Correct order (0.5 each = 2.5, all-correct sequence 4):

output = model(x)
loss = criterion(output, y)
optimizer.zero_grad()
loss.backward()
optimizer.step() (4) (zero_grad may also legally precede forward; accept.) Why zero_grad(): PyTorch accumulates gradients into .grad; without clearing, gradients from previous batches add up and corrupt the update. (1)

Q7. (5 marks) (a) device = torch.device("cuda" if torch.cuda.is_available() else "cpu") (1) (b) Operations require operands on the same device; a model on GPU with input on CPU raises a runtime error (cannot mix devices in a single op). (2) (c) Returns a new tensor on the target device (for tensors, .to() is not in place — you must reassign x = x.to(device)). Note: for nn.Module, .to() moves parameters in place. (2)

Q8. (4 marks) (a) state_dict() — a dictionary mapping parameter names to tensors. (1) (b)

torch.save(model.state_dict(), "ckpt.pth")
model.load_state_dict(torch.load("ckpt.pth"))

(2) (c) It is more portable/robust — avoids pickling class definitions/paths, decoupling weights from code structure. (1)

Q9. (3 marks — 1 each)

Build/define the model — Sequential([...]) (add layers).
Compile — model.compile(optimizer, loss, metrics).
Train — model.fit(x, y, epochs, batch_size).

Q10. (3 marks) (a) In FP16, small gradients underflow to zero; the scaler multiplies the loss by a large factor before backward (and unscales before the step) to keep gradients in representable range and preserve numerical stability. (2) (b) TensorBoard (or Weights & Biases). (1)

[
  {"claim":"Q2 gradient of y=x1^2+x2^2 at (2,3) is [4,6]","code":"x1,x2=symbols('x1 x2'); y=x1**2+x2**2; g=[diff(y,x1).subs({x1:2}), diff(y,x2).subs({x2:3})]; result = g==[4,6]"},
  {"claim":"Q2 general derivative dy/dxi = 2xi","code":"xi=symbols('xi'); result = diff(xi**2, xi)==2*xi"},
  {"claim":"Loss scaled by factor S has gradient scaled by S (linearity)","code":"S,g=symbols('S g'); L=symbols('L'); result = diff(S*L, L)==S"}
]