Visual walkthrough — Continual and lifelong learning

Before we start, two words we lean on constantly:

---

Step 1 — One task: walk downhill to the valley floor

WHAT. We train Task 1 alone. Starting from a random knob setting, we repeatedly nudge $\theta$ in the steepest-downhill direction of Task 1's loss until we can't go lower.

WHY. "Steepest downhill" has a name: the negative gradient. The gradient ${\nabla }_{\theta }{\mathcal{L}}_1$ (the symbol $\nabla$, "nabla", just means "slope in every knob-direction at once") points uphill; so the negative gradient points to where the loss drops fastest. That is exactly the tool we want when the only question is "which way is down?" — no other tool answers that directly.

$\theta \leftarrow \theta -\underset{\begin{array}{c}\text{step size:}\\ \text{how far we walk}\end{array}}{\underbrace{\eta }}\underset{\begin{array}{c}\text{uphill direction}\\ \text{of Task 1’s loss}\end{array}}{\underbrace{{\nabla }_{\theta }{\mathcal{L}}_1(\theta )}}$

The little "$\leftarrow$" means "replace the left with the right". We repeat until we land at the valley floor, which we name ${\theta }_1^{\ast }$ (the star means "best for Task 1").

PICTURE. The bowl below is Task 1's loss. The white dot rolls down to the bottom ${\theta }_1^{\ast }$. At the very bottom the ground is flat — the slope is zero.

---

Step 2 — Task 2 arrives, and it pulls us off the floor

WHAT. Task 1's data is gone. Task 2 shows up with its own landscape and its own valley ${\theta }_2^{\ast }$, sitting somewhere else on the map. Standard SGD does one thing: it walks downhill on Task 2, starting from ${\theta }_1^{\ast }$.

WHY. SGD only ever knows the loss you feed it. Right now that is ${\mathcal{L}}_2$ only. So its gradient ${\nabla }_{\theta }{\mathcal{L}}_2({\theta }_1^{\ast })$ points toward Task 2's valley — with zero regard for where Task 1's valley was.

$\theta \leftarrow {\theta }_1^{\ast }-\eta {\nabla }_{\theta }{\mathcal{L}}_2({\theta }_1^{\ast })$

Every symbol is the same as Step 1; only the subscript changed from $1$ to $2$. That tiny change is the whole disaster: we start on Task 1's floor but march toward Task 2's floor.

PICTURE. Two bowls on one map. The dot starts in the left bowl's bottom (${\theta }_1^{\ast }$) and gets tugged (orange arrow) toward the right bowl's bottom (${\theta }_2^{\ast }$).

---

Step 3 — See the forgetting: climbing out of Task 1's valley

WHAT. As we walk toward ${\theta }_2^{\ast }$, watch what happens to Task 1's loss — the thing we stopped measuring.

WHY. We left Task 1's floor. Any step away from the bottom of a bowl means walking uphill on that bowl. So ${\mathcal{L}}_1$ silently grows while we're busy minimising ${\mathcal{L}}_2$. Nobody is watching Task 1, so nobody stops it. That rising ${\mathcal{L}}_1$ is catastrophic forgetting.

PICTURE. Same map, but now we overlay Task 1's height as a colour. The path from ${\theta }_1^{\ast }$ to ${\theta }_2^{\ast }$ climbs out of the blue (low ${\mathcal{L}}_1$) into the hot (high ${\mathcal{L}}_1$) zone. The reader can literally trace the loss going up.

Two root causes, now visible:

• Parameter overlap — the same $\theta$ axes are shared by both bowls.
• No memory — once ${\mathcal{D}}_1$ is gone, nothing pins us to the left bowl.

---

Step 4 — The wish: don't leave the valley along its steep walls

WHAT. We want to reach Task 2's valley while not climbing far up Task 1's bowl. The trick: bowls are not equally steep in all directions. Some directions are steep cliffs (moving there wrecks Task 1); others are gentle plains (moving there barely changes ${\mathcal{L}}_1$).

WHY. If we could measure the steepness of Task 1's bowl in each knob-direction, we could tell SGD: "you may move freely along the gentle directions, but resist the steep ones." That measurement is exactly what we build next.

PICTURE. A single valley drawn as an elongated oval (an ellipse). One axis is a steep wall (small budge = big loss jump), the perpendicular axis is a gentle floor (big budge = tiny loss jump). Arrows label "cheap to move" vs "expensive to move".

---

Step 5 — Approximate the bowl with a parabola (Taylor's tool)

WHAT. We don't have Task 1's data anymore, so we can't redraw its whole bowl. Instead we approximate its shape near the bottom with the simplest curved bowl there is: a parabola.

WHY this tool. Near a smooth minimum, every landscape looks like a parabola — that's what a Taylor expansion (a "zoom-in polynomial" that copies a function's value, slope, and curvature at a point) tells us. We use it because it needs only what we can still afford to store: the location of the floor and its curvature. The linear (slope) term vanishes because — recall Step 1 — the slope at the floor is zero. So:

${\mathcal{L}}_1(\theta )\approx \underset{\text{height of the floor (a constant)}}{\underbrace{{\mathcal{L}}_1({\theta }_1^{\ast })}}+\underset{\text{the parabola’s curved cup}}{\underbrace{\frac{1}{2}(\theta -{\theta }_1^{\ast }{)}^{\top }H(\theta -{\theta }_1^{\ast })}}$

Reading it term by term:

• $(\theta -{\theta }_1^{\ast })$ — how far we strayed from Task 1's floor, one number per knob.
• $H$ — the Hessian, a table of curvatures. Formally, entry $(i,j)$ of $H$ is the second derivative of the loss with respect to two knobs at once: $H_{ij}=\frac{{\partial }^2{\mathcal{L}}_1}{\partial {\theta }_i\partial {\theta }_j}{|}_{{\theta }_1^{\ast }}$ Here ${\partial }^2/\partial {\theta }_i\partial {\theta }_j$ means "differentiate twice — once for knob $i$, once for knob $j$"; a second derivative measures curvature (how fast the slope itself changes). So a big diagonal entry $H_{jj}$ = steep cliff along knob $j$; a small one = gentle plain. This is the "steepness meter" we wished for in Step 4.
• $\frac{1}{2}(\cdot {)}^{\top }H(\cdot )$ — combines stray-distance with steepness into the extra loss you pay. The "${}^{\top }$" ("transpose") and the sandwich are just the multi-knob way of writing $\frac{1}{2}\times \text{curvature}\times {\text{distance}}^2$.

PICTURE. Task 1's true bowl (solid) with the fitted parabola (dashed) hugging it near ${\theta }_1^{\ast }$ — a perfect match at the bottom, drifting apart far away (which is fine; we only stay near the bottom).

---

Step 6 — Replace the full curvature table with Fisher (keep only the diagonal)

WHAT. The full Hessian $H$ is a giant square table (millions × millions of entries). We can't store it. So we keep only its diagonal — one steepness number per knob — and we estimate those numbers with the Fisher Information.

First, the model's guess $p(y\mid \mathbf{x};\theta )$. A classifier doesn't just blurt one answer; it outputs a confidence for each possible answer — a predictive distribution written $p(y\mid \mathbf{x};\theta )$, read "the probability the model assigns to answer $y$ given input $\mathbf{x}$, using knobs $\theta$." For example it might say "cat: 0.9, dog: 0.1". We look at $p$ (rather than the raw loss) because the Fisher trick below asks a probabilistic question: how sharply does the model's confidence in the true answer react when we jiggle a knob? A knob that barely moves the confidence was unimportant; a knob that swings it a lot was critical.

WHY Fisher, and why the diagonal. We compute the Fisher Information $F_j$ cheaply from the old model's own predictions on ${\mathcal{D}}_{t-1}$:

$F_j={\mathbb{E}}_{{\mathcal{D}}_{t-1}}\left[{\left(\frac{\partial \log p(y\mid \mathbf{x};\theta )}{\partial {\theta }_j}\right)}^2\right]$

• ${\mathbb{E}}_{{\mathcal{D}}_{t-1}}[\cdot ]$ — "average over the examples of task $t-1$" ($\mathbb{E}$ is the average, or expectation, symbol).
• $\log p(y\mid \mathbf{x};\theta )$ — how confident the model is in the correct answer (the log just turns multiplying probabilities into adding, which is friendlier).
• $\partial (\cdot )/\partial {\theta }_j$ — how much that confidence wobbles if you jiggle knob $j$.
• squared and averaged — big $F_j$ means "knob $j$ strongly affects the old task's predictions" = important knob = steep wall. Small $F_j$ = gentle plain. This is precisely our per-direction steepness meter.

Why Fisher $\approx$ Hessian diagonal — and under which assumptions. The link is exact only when three conditions hold: (i) the loss is the negative log-likelihood $-\log p(y\mid \mathbf{x};\theta )$ (standard cross-entropy for classification); (ii) we sit at the converged minimum ${\theta }_{t-1}^{\ast }$, where the gradient is zero (Step 1); and (iii) the model is well-specified — its predicted distribution matches the data's true label distribution, so the labels $y$ used to average are drawn from the model itself. Under (i)–(iii) the expected Hessian equals the Fisher matrix, and its diagonal is our $F_j$. In practice these hold only approximately, which is why EWC's penalty is an approximation, not an exact copy, of Task $t-1$'s bowl.

Keeping only the diagonal means we approximate the tilted oval of Step 4 by an axis-aligned oval — a small inaccuracy for a huge memory saving.

PICTURE. The tilted ellipse of Step 4 alongside its axis-aligned Fisher approximation, with each axis stamped by its $F_j$: a large $F$ on the steep axis, a tiny $F$ on the gentle one.

---

Step 7 — Assemble EWC: two bowls, one compromise

WHAT. Add the parabola-approximation of task $t-1$'s bowl as a penalty on top of task $t$'s real loss (remember from Step 6: $t-1$ is the finished task, $t$ the new one):

${\mathcal{L}}_t^{\text{EWC}}(\theta )=\underset{\begin{array}{c}\text{new task’s real bowl}\\ \text{(pulls toward }{\theta }_t^{\ast })\end{array}}{\underbrace{{\mathcal{L}}_t(\theta )}}+\frac{\lambda }{2}{\sum }_j\underset{\begin{array}{c}\text{steepness of old}\\ \text{task in knob }j\end{array}}{\underbrace{F_j}}\underset{\begin{array}{c}\text{how far knob }j\\ \text{strayed, squared}\end{array}}{\underbrace{({\theta }_j-{\theta }_{t-1,j}^{\ast }{)}^2}}$

Term by term:

• ${\mathcal{L}}_t(\theta )$ — the plasticity term: still wants to reach the new task's valley.
• $F_j({\theta }_j-{\theta }_{t-1,j}^{\ast }{)}^2$ — a spring on each knob. Its stiffness is $F_j$: steel spring on important knobs, loose spring on unimportant ones.
• $\lambda$ (lambda) — one global dial for how stiff all the springs are. Big $\lambda$ = cling to the old task (stability); small $\lambda$ = freely chase the new task (plasticity). This is the stability–plasticity dilemma made into a knob.

WHY it works. The total landscape is now the sum of two bowls. Its new minimum sits between ${\theta }_{t-1}^{\ast }$ and ${\theta }_t^{\ast }$ — pulled toward the new task, but resisting exactly along the old task's steep directions. We reach the new task through the gentle directions of the old task, so its loss barely rises.

PICTURE. The old task's Fisher-parabola (contours) plus the new task's bowl. Their sum has a new minimum ${\theta }_{\text{EWC}}^{\ast }$ that slides toward the new task along the gentle axis, staying near the old floor on the steep axis. Compare its path to the naive orange path of Step 2 — the EWC path curves to hug low old-task loss.

---

Step 8 — The degenerate cases (don't get surprised)

WHAT & WHY. Every dial has extremes; here is what the landscape does at each.

• $\lambda =0$. The springs vanish → EWC collapses to plain SGD → full forgetting (back to Step 2). Picture: only the new task's bowl remains.
• $\lambda \to \infty$. Springs infinitely stiff → knobs frozen at ${\theta }_{t-1}^{\ast }$ → new task never learned. Perfect memory, zero new skill.
• $F_j=0$ for some knob. That direction was irrelevant to the old task (flat plain) → no spring → new task uses it freely, at no cost. Correct behaviour.
• Tasks share a valley (${\theta }_{t-1}^{\ast }\approx {\theta }_t^{\ast }$). Both bowls overlap; springs cost almost nothing at ${\theta }_t^{\ast }$. This is easy forward/backward transfer — no conflict to resolve.
• Many tasks ($t=3,4,\dots$). The penalty accumulates: each finished task leaves its own set of springs. Naively they can over-stiffen the whole net (everything becomes important). This is the known limitation — later methods (online-EWC) merge the springs so they don't pile up unboundedly.

---

The one-picture summary

The whole derivation on one map: the old task's bowl (frozen shape, kept as a Fisher-parabola), the new task's bowl (real), naive SGD's forgetting path (orange, climbs out of the old task), and EWC's compromise path (violet, curves along the gentle direction to ${\theta }_{\text{EWC}}^{\ast }$).

Recall Feynman retelling (say it out loud)

Training a network is rolling a ball down into a valley — the bottom is "knobs that do the old task well." When a new task arrives with its own valley somewhere else, rolling toward it climbs us straight out of the old valley, and that climb is forgetting. We can't keep the old task's whole map, but near its bottom the valley is just a curved cup we can copy with a parabola. The parabola's steepness in each knob-direction — how much the old task cares about that knob — is measured by Fisher. We then bolt a spring onto every knob: stiff where the old task was steep, loose where it was flat. Now reaching the new task means sliding through the loose directions and resisting the stiff ones, so we settle at a compromise point that does both. One dial $\lambda$ sets how hard the springs pull: zero springs = forget everything, infinite springs = learn nothing new.

Recall Self-test

Why does the linear term drop out of the old task's Taylor expansion? ::: Because at the valley floor ${\theta }_{t-1}^{\ast }$ the gradient (slope) is zero, so the first-order term is zero. What does a large Fisher value $F_j$ mean geometrically? ::: The old task's loss bowl is steep along knob $j$; moving that knob badly hurts the old task, so EWC puts a stiff spring there. What happens to EWC as $\lambda \to \infty$? ::: The knobs freeze at ${\theta }_{t-1}^{\ast }$: perfect stability, zero plasticity (the new task is never learned). In one sentence, why does naive SGD forget? ::: It only measures the current task's loss, so it walks toward the new valley and climbs out of the old one with nothing watching the old loss. Under which key assumption does the Fisher Information equal the diagonal of the Hessian? ::: When the loss is the negative log-likelihood and we evaluate at the converged minimum of a well-specified model (labels drawn from the model itself).

See also: Catastrophic Forgetting · Regularization Techniques · Knowledge Distillation · Online Learning · Neuroscience - Memory Consolidation · Meta-Learning · Neural Architecture Search