Backpropagation through time: Question bank — AI-ML deep dive

Intuition How to use this page

Each line below hides its answer. Read the prompt, commit to an answer out loud with a reason, then reveal. If your reason was wrong even when your yes/no was right, you have found a real gap. These items target the misconceptions that BPTT invites — the sneaky sign, the forgotten factor, the "which weight" confusion.

Reminders of the notation we test (all built in the parent):

$W_{hh}$ = hidden-to-hidden (the shared recurrent weight), $W_{x h}$ = input-to-hidden, $W_{h y}$ = hidden-to-output.
$h_{t}$ = the hidden state after the $tanh$ (post-activation); $a_{t}$ = the linear sum before the $tanh$ (pre-activation), so $h_{t} = tanh (a_{t})$ .
$δ_{t} = \partial L / \partial h_{t}$ (post-activation gradient); $\hat{δ}_{t} = δ_{t} ⊙ (1 - h_{t}^{2})$ (pre-activation gradient).
$⊙$ means elementwise multiply; $diag (v)$ is a matrix with $v$ on its diagonal.

True or false — justify

The recurrent weight

W_{hh}

has one gradient per time step that you keep separate.

False.

W_{hh}

is shared across all steps, so you sum the per-step contributions into a single gradient

\partial L / \partial W_{hh} = \sum_{t} \hat{δ}_{t} h_{t - 1}^{T}

Unrolling an RNN over

T

steps turns it into a feedforward net whose layers have independent weights.

False. The unrolled layers all reuse the same

W_{hh}

W_{x h}

— that shared weight is exactly why gradients must be summed rather than treated separately.

In BPTT the gradient

δ_{t}

at step

t

depends only on the loss

L_{t}

produced at that step.

False.

δ_{t}

has two sources: the local loss

L_{t}

and the future loss carried back through

h_{t + 1}

δ_{t} = \partial L_{t} / \partial h_{t} + W_{hh}^{T} diag (1 - h_{t + 1}^{2}) δ_{t + 1}

The backward pass can be run left-to-right at the same time as the forward pass.

False.

δ_{t}

needs

δ_{t + 1}

, which needs future hidden states. You must finish the forward pass, then sweep backward from

t = T

down to

t = 1

The Jacobian

\partial h_{t + 1} / \partial h_{t}

equals

W_{hh}

alone.

False. It is

diag (1 - h_{t + 1}^{2}) W_{hh}

— the

tanh

derivative

(1 - h_{t + 1}^{2})

multiplies element-by-element before

W_{hh}

. Dropping it ignores the nonlinearity.

For a sequence of length

T

, BPTT stores all

T

hidden states in memory during the forward pass.

True. Each

h_{t}

(and the pre-activations) is needed in the backward pass to compute local gradients, so memory grows linearly with

T

— motivating truncated BPTT on long sequences.

W_{x h}

(input weights) also needs its gradient summed over all time steps.

True. Like

W_{hh}

W_{x h}

is reused every step, so

\partial L / \partial W_{x h} = \sum_{t} \hat{δ}_{t} x_{t}^{T}

— same summation logic.

The factor

(1 - h_{t}^{2})

can never make a gradient vanish because it is always positive.

Half-true, and dangerous. It is in

(0, 1]

, so positive — but values well below 1 multiply across many steps and shrink the product, one root cause of vanishing gradients.

Spot the error

A student writes

\partial L / \partial W_{hh} = \sum_{t} δ_{t} h_{t - 1}^{T}

. What is missing?

The

tanh

factor. It must be

\hat{δ}_{t} h_{t - 1}^{T} = (δ_{t} ⊙ (1 - h_{t}^{2})) h_{t - 1}^{T}

—

δ_{t}

is post-activation, so you still have to pass through the

tanh

derivative to reach the pre-activation

a_{t}

where

W_{hh}

lives.

Someone claims the local

W_{hh}

gradient at step

t

multiplies by

h_{t}

(the current hidden state). Correct partner?

Wrong state — it multiplies by

h_{t - 1}

, the incoming hidden state, because

a_{t} = W_{hh} h_{t - 1} + \dots

, so

\partial a_{t} / \partial W_{hh} = h_{t - 1}^{T}

In the backward recursion a student uses

W_{hh}

(not transposed):

δ_{t} = \dots + W_{hh} diag (1 - h_{t + 1}^{2}) δ_{t + 1}

. Fix?

Use

W_{hh}^{T}

. Forward multiplies by

W_{hh}

; the backward (adjoint) direction transposes it, so gradients flow through

W_{hh}^{T}

A note says "

h_{2} - 0.3

is the gradient of

L

w.r.t.

W_{hh}

at step 2." Diagnose.

That is only

δ_{2} = \partial L_{2} / \partial h_{2}

. To get the weight gradient you still multiply by

(1 - h_{2}^{2})

and by

h_{1}

(h_{2} - 0.3) (1 - h_{2}^{2}) h_{1}

A learner applies the chain rule but forgets the direct effect of

W_{hh}

h_{t}

, keeping only the path through

h_{t - 1}

. Consequence?

The gradient is under-counted.

\partial h_{t} / \partial W_{hh}

has a direct term (this step) plus the indirect term through

h_{t - 1}

; dropping the direct term loses this step's own contribution.

Someone computes

\partial L_{2} / \partial W_{hh}

2 \times

the true value in Worked Example 1 (

0.31

instead of

0.155

). Likely cause?

They omitted the

(1 - h_{2}^{2}) \approx 0.5

factor. Since it was near

0.5

here, dropping it roughly doubled the result — the classic BPTT bug.

Why questions

Why do we sum

\partial L / \partial W_{hh}

over time instead of averaging or taking the last step?

Because the same parameter influences the loss at every step; the total derivative is the sum of all these separate influences (multivariate chain rule), not one representative.

Why is "backpropagation through time" the same algorithm as ordinary backpropagation, not a new one?

Once you unroll the RNN, it is a deep feedforward graph; you run standard backprop on it. The only twist is weight sharing forces the gradient summation.

Why does the Jacobian

\partial h_{t + 1} / \partial h_{t}

use a diagonal matrix for the

tanh

term?

tanh

acts elementwise — output component

i

depends only on input component

i

— so its derivative is a diagonal matrix, with no cross-terms between hidden units.

Why does a long sequence make gradients tend to vanish or explode?

The backward chain multiplies

W_{hh}^{T} diag (1 - h^{2})

once per step; over many steps this repeated product shrinks toward zero (spectral norm

< 1

) or blows up (

> 1

), the vanishing/exploding problem.

Why do LSTMs and GRUs help BPTT compared to a plain

tanh

RNN?

They add a gated, near-additive memory path so the backward product stays near 1 across steps, avoiding the repeated shrinking

(1 - h^{2})

factors that kill vanilla-RNN gradients.

Why is

\hat{δ}_{t} = δ_{t} ⊙ (1 - h_{t}^{2})

called the pre-activation gradient?

Because it is

\partial L / \partial a_{t}

— the gradient at the linear sum

a_{t}

before the

tanh

— obtained by pushing the post-activation

δ_{t}

back through the

tanh

derivative.

Edge cases

What happens to the

W_{hh}

gradient contribution at

t = 1

when

h_{0} = 0

It vanishes: the step-1 term is

\hat{δ}_{1} h_{0}^{T} = \hat{δ}_{1} \cdot 0 = 0

. With a zero initial state the first step contributes nothing to

\partial L / \partial W_{hh}

If a hidden unit saturates so

h_{t} \approx \pm 1

, what does

(1 - h_{t}^{2})

do to its gradient?

It approaches

0

, so that unit's gradient is squeezed to near zero — the neuron stops learning ("dead" saturated

tanh

), a local cause of vanishing gradients.

For the very last step

t = T

, why is there no future term in

δ_{T}

There is no

h_{T + 1}

L_{T + 1}

, so the recursion has no future contribution:

δ_{T} = \partial L_{T} / \partial h_{T}

alone. It is the base case of the backward sweep.

If the output uses identity (

y_{t} = h_{t}

) instead of a

W_{h y}

layer, does BPTT change?

Only the local term

\partial L_{t} / \partial h_{t}

simplifies (no extra

W_{h y}^{T}

factor). The recurrence through time and the weight-summation are unchanged.

For a length-1 sequence (

T = 1

), does BPTT reduce to ordinary backprop?

Yes. With one step there is no time chain —

δ_{1} = \partial L_{1} / \partial h_{1}

and

\partial L / \partial W_{hh} = \hat{δ}_{1} h_{0}^{T}

— identical to a single-layer feedforward backprop.

In a bidirectional RNN, is there one backward sweep or two?

Two independent BPTT sweeps — one for the forward-direction chain and one for the backward-direction chain — since each direction has its own hidden states and shared weights.

Recall Quick self-test

Name the two sources that feed $δ_{t}$ . ::: The local loss $\partial L_{t} / \partial h_{t}$ and the future gradient $W_{hh}^{T} diag (1 - h_{t + 1}^{2}) δ_{t + 1}$ . Which hidden state multiplies the $W_{hh}$ gradient at step $t$ ? ::: $h_{t - 1}$ (the incoming one). What single factor turns $δ_{t}$ into $\hat{δ}_{t}$ ? ::: $(1 - h_{t}^{2})$ , the $tanh$ derivative.