Training loops from scratch: Question bank — AI-ML deep dive

Before we start, three symbols we lean on (all built in Training loops from scratch):

$θ$ = the model's parameters (all its weights and biases stacked together).
$η$ (Greek "eta") = the learning rate, the size of each step we take.
$\nabla_{θ} L$ = the gradient, an arrow pointing in the direction that increases loss fastest. We step the opposite way.

True or false — justify

A training loop always converges if you run enough epochs.

False. Gradient descent can oscillate around a minimum, get stuck at a saddle point, or diverge entirely if

η

is too large — more epochs don't fix a bad learning rate.

The loss must decrease at every single step.

False. Mini-batch gradients are noisy estimates, so any one batch can push loss up; only the trend over many steps is expected to fall. See 4.2.01-Gradient-descent-variants.

Averaging the loss over a batch instead of summing changes only the number displayed, not the learning.

False. Sum vs. mean scales the gradient by the batch size

B

, which is equivalent to multiplying

η

B

— it directly changes the effective step size.

Shuffling the data each epoch is just a cosmetic nicety.

False. Without shuffling, consecutive batches are correlated (e.g. all class-0 then all class-1), biasing each gradient step and producing zig-zag, poorly-generalizing updates.

loss.backward() also updates the weights.

False. backward() only fills in the gradients

\nabla_{θ} L

; the weights change only when optimizer.step() applies the update rule. Two separate jobs.

Calling optimizer.zero_grad() erases the model's learned weights.

False. It zeroes the accumulated gradient buffers, not the parameters. The weights

θ

are untouched; only the "how to change them next" slate is wiped.

A lower training loss always means a better model.

False. Training loss can keep falling while validation loss rises — that gap is overfitting, memorizing noise instead of learning the pattern.

Empirical risk (average loss over your data) is the quantity we actually care about.

False. We care about expected risk on unseen data from the true distribution; empirical risk is only a proxy we can compute, and minimizing it too hard causes overfitting.

Using the full dataset for every gradient step is strictly more correct than mini-batches.

Half-true. Full-batch gives the exact gradient but is slow and its low noise can trap you in sharp minima; the noise in mini-batches often helps generalization, so "more correct gradient" ≠ "better training."

The chain rule and backpropagation are two different algorithms.

False. Backprop is the chain rule applied efficiently across layers, reusing intermediate results so each derivative is computed once. See 3.1.02-Backpropagation-algorithm.

Spot the error

loss.backward()
optimizer.step()
# next batch...
loss.backward()

What breaks?

optimizer.zero_grad() is missing. PyTorch accumulates gradients, so the second backward() adds to the first — the update uses a stale sum of two batches, causing exploding, wrong-direction steps.

for epoch in range(50):
    for X_batch, y_batch in loader:
        y_pred = model(X_batch)
        loss = criterion(y_pred, y_batch)
        optimizer.zero_grad()
        optimizer.step()
        loss.backward()

What is out of order?

loss.backward() must come before optimizer.step(). Here step() runs on zeroed (empty) gradients, so nothing is learned, and backward() afterwards is wasted.

W2 -= lr * dW2      # updated
db2 = np.sum(dloss_dy)   # gradient computed AFTER using it

Why is this a bug in a from-scratch loop?

A gradient must be computed from the current weights before any update. Interleaving update and gradient computation corrupts the chain-rule values that later layers depend on — compute all gradients first, then update all.

z1 = X_batch @ W1 + b1
dz1 = da1 * (z1 > 0)

A student replaces (z1 > 0) with (a1 > 0). Problem?

For ReLU with

a_{1} = max (0, z_{1})

this happens to agree, but the concept is wrong: the ReLU derivative is "1 where the pre-activation

z_{1} > 0

." If the activation ever produced exact zeros differently, using

a_{1}

hides which path was clamped.

lr = 10.0

Someone sets a huge learning rate to "train faster." Likely outcome?

The step overshoots the minimum each time, so loss grows without bound (diverges) — often printed as nan. Bigger

η

is faster only up to a threshold; see 3.4.03-Learning-rate-schedules.

W1 = np.zeros((D, 4))
W2 = np.zeros((4, 1))

Why won't this network learn?

All-zero weights make every hidden unit compute the same thing and receive the same gradient — perfect symmetry. Units never differentiate. That's why we use small random init.

Why questions

Why do we move opposite to the gradient, not along it?

The gradient points toward the steepest increase in loss; we want to decrease loss, so we step in the negative-gradient direction.

Why divide the batch gradient by

B

(batch size)?

To get the average per-example gradient, making the update magnitude independent of how many examples happen to be in the batch — otherwise

η

silently changes with batch size.

Why initialize weights small (e.g.

\times 0.1

) rather than large?

Large weights push activations into saturated or huge regions, giving tiny or exploding gradients at the start; small init keeps signals and gradients in a healthy range so learning can begin.

Why is MSE a convenient loss to differentiate?

Its derivative is simply

2 (\overset{y}{^} - y) / B

— smooth, defined everywhere, and proportional to the error, giving a clean learning signal. See 3.3.06-Custom-loss-functions.

Why does ReLU avoid the vanishing-gradient trouble that squashing functions cause?

For positive inputs its derivative is exactly 1, so gradients pass through undiminished, whereas sigmoids flatten to near-zero slope and choke the backward signal.

Why do we iterate over many epochs instead of one perfect pass?

Each epoch's noisy mini-batch steps only partially reduce loss; repeating lets the parameters settle toward the minimum and lets us watch validation loss to catch overfitting.

Why does batch normalization change how the loop behaves?

It renormalizes activations per batch, smoothing the loss landscape so larger, more stable steps are possible — but it makes each example's output depend on its batch-mates. See 3.4.01-Batch-normalization.

Edge cases

Batch size equals the whole dataset — what have you built?

Full-batch gradient descent: the gradient is exact and deterministic, with zero mini-batch noise, but each step is maximally expensive.

Batch size equals 1 — what is this called and what's the downside?

Pure stochastic gradient descent: gradients are extremely noisy, updates jitter a lot, and you lose vectorized speed — but the noise can help escape sharp minima.

The dataset size

N

is not divisible by the batch size — what happens to the last batch?

You get a smaller final batch. If you average the loss it's fine; if you sum, that batch's effective step is smaller, so prefer averaging or drop the remainder for uniformity.

A ReLU unit receives only negative inputs for many batches — what dies?

A "dead ReLU": its output and derivative are both 0, so no gradient flows and its incoming weights never update. It stays dead unless something shifts its inputs positive.

The gradient comes out exactly zero everywhere — are you done?

Not necessarily. Zero gradient occurs at minima and at maxima and saddle points; you must check whether loss is actually low, not just flat.

Loss suddenly prints nan mid-training — most common cause?

Numerical explosion: too-large

η

, un-clipped gradients, or a log/divide on an invalid value. The loop faithfully propagates the blow-up; fix the learning rate or clip gradients.

Recall Self-test

The two lines that must never be swapped or skipped around an update? ::: optimizer.zero_grad() then loss.backward() then optimizer.step() — clear, compute, apply, in that order. The one-word reason a falling training loss can still mean a worse model? ::: Overfitting.