Foundations — Backpropagation through time

This page assumes you have seen nothing. Before you touch Backpropagation through time, you must own every letter it prints. We collect them here, define each in plain words, draw the picture it stands for, and say why the topic can't live without it. Read top to bottom — each item leans on the one above it.

0. The two worlds we are bridging

The parent topic sits between two ideas you may already have met: an RNN (the thing we differentiate) and the backpropagation algorithm (the tool we differentiate with). BPTT is literally "backprop, applied to an RNN that has been flattened out in time."

The left drawing is the RNN as we store it: one box with an arrow looping back into itself. The right drawing is the same network unrolled — copied once per time step, the loop straightened into a left-to-right chain. Every symbol below describes a piece of that right-hand picture.

1. Time steps and the index $t$

• Plain words: $t$ is just "which slot am I looking at."
• Picture: in the unrolled diagram (Figure 1, right), $t$ counts the copies from left to right.
• Why the topic needs it: every quantity below carries a subscript $t$ because it has a different value at every moment. Without $t$ we couldn't say "the hidden state now versus one step ago."

$T$ (capital) is the length of the whole sequence — the number of copies in the unrolled chain.

2. Vectors, and the input $x_t$

• Picture: a tall stack of numbered boxes. If a word is turned into 3 numbers, its vector is 3 boxes tall.
• $x_t$ = the input at time $t$: the vector fed in at that slot (e.g. the numbers standing for the $t$-th word).
• Why needed: the network reads one $x_t$ per step. It's the fresh information arriving at each copy in the chain.

3. The hidden state $h_t$ — the network's memory

• Picture (Figure 2): the horizontal arrows in the unrolled chain. $h_{t-1}$ flows into the box at step $t$, gets mixed with $x_t$, and out comes $h_t$, which flows into step $t+1$.
• $h_0$ is the memory before any input — usually all zeros (the network starts blank).
• Why needed: because $h_t$ depends on $h_{t-1}$, which depends on $h_{t-2}$… changing anything early ripples through all later states. That ripple is exactly what BPTT must chase backward.

4. Weight matrices: $W_{hh}$, $W_{xh}$, $W_{hy}$

The parent uses three, and mixing them up is the classic beginner error:

• Picture: in Figure 2, $W_{xh}$ labels the arrow up from $x_t$, $W_{hh}$ labels the arrow across from $h_{t-1}$, $W_{hy}$ labels the arrow up to $y_t$. Same three labels repeat on every copy — that repetition is why gradients must be added across time.
• $b_h$ (bias): an extra constant vector added in each box, a fixed nudge independent of the inputs.
• Why needed: the whole point of BPTT is finding how to change $W_{hh}$ (and friends). It is the one shared across all copies, so its gradient is a sum over $t$.

5. $\tanh$ — the squashing function

• Picture (Figure 3, blue curve): an S-shape that flattens at $\pm 1$. It is applied element by element — each number in the pre-activation vector goes through the same S.
• Why this tool and not another? The network needs a nonlinearity (otherwise stacking layers just makes one big matrix). $\tanh$ is smooth (so we can differentiate it — essential for backprop) and centred at zero (keeps the memory from drifting).
• Its derivative is the yellow bump in Figure 3: $\frac{d}{da}\tanh (a)=1-{\tanh }^2(a)$. Read it as: slope of the S-curve. It is tallest ($=1$) at the middle and shrinks toward $0$ at the flat ends.

6. Pre-activation $a_t$ vs post-activation $h_t$

• Picture: inside each box in Figure 2 there are two stages — first add up the weighted inputs ($a_t$), then squash ($h_t$). $a_t$ is the "raw score," $h_t$ is the "polished memory."
• Why needed: the weights $W_{hh},W_{xh}$ live before the squash, in $a_t$. So to reach them from the loss we must pass back through the $\tanh$ slope — the exact source of the $(1-h^2)$ term.

7. Output $y_t$, target ${\hat{y}}_t$, and loss $L_t$, $L$

• $y_t$ = the network's prediction at step $t$ (from $y_t=W_{hy}{h}_t+b_y$).
• ${\hat{y}}_t$ (with a hat) = the target, the true answer we wanted.
• Picture: at each copy in the chain a small "scorecard" $L_t$ compares $y_t$ to ${\hat{y}}_t$. The grand total is the sum down the chain.
• Why $\Sigma$ (the sum)? Because there is a mistake at every time step, and the network should improve on all of them. The symbol ${\sum }_{t=1}^T$ just means "add these up as $t$ runs from $1$ to $T$."

8. Derivatives, the chain rule, and ${\delta }_t$

• Picture (Figure 4): the sensitivity is the slope of the loss-vs-weight curve at our current point. Steep slope → big correction.

• Why this tool? In an unrolled RNN, a weight affects the loss only through a long chain of hidden states. The chain rule multiplies the link-by-link slopes; when several chains reach the same weight, we add the chains. That "multiply along, add across" is literally all of BPTT.

• Picture: while the blue forward arrows carry memory rightward, the red ${\delta }_t$ arrows carry blame leftward. Each ${\delta }_t$ collects blame from its own step ($\partial L_t/\partial h_t$) plus whatever the future (${\delta }_{t+1}$) passes back.
• Extra symbols you'll meet: $\odot$ = elementwise product (multiply matching entries of two equal-size vectors), $W^{\mathsf{T}}$ = transpose (flip the grid so it points the gradient the right way), $\text{diag}(v)$ = a matrix with vector $v$ on its diagonal and zeros elsewhere (the elementwise $\tanh$-slope, dressed as a matrix).

Prerequisite map

Every arrow above is a "you need this before that." Trace any path to the bottom box — that path is a prerequisite thread for the parent topic.

Equipment checklist

Cover the right side; can you answer before revealing?