Intuition The One Core IdeaA neural network is a big list of numbers (its "weights"). When you teach it a new task, you nudge those numbers , but the very same numbers were also holding the memory of the old task — so nudging them for the new lesson quietly erases the old one. Continual learning is the art of nudging the numbers for the new task while protecting the ones that still remember the old tasks.
This page assumes you have never seen a single symbol from the parent note. We build each one from nothing, anchor it to a picture, and only then use it. Read top to bottom — nothing appears before it is earned.
Before any Greek letters, picture a machine with knobs.
A model is a machine that takes an input (a picture, a sentence) and produces an output (a label like "cat"). Inside, it has thousands of little knobs . Turning the knobs changes what output you get. Learning = finding the right knob settings.
θ θ "the current settings of all the knobs, written as one long list of numbers."
Picture: the row of dials in the figure above.Why the topic needs it: every task the model learns is stored in these numbers . Forgetting happens because the knobs get re-turned. You cannot talk about forgetting without a name for "the thing that gets overwritten."
When we write θ = [ w 1 , w 2 , b ] w b θ j j "knob number j — a single dial picked out of the list.
x y
==x one input example , e.g. the pixels of one photo. Bold means it is a list of numbers (many pixels), not a single number.
==y the correct answer for that input, e.g. the label "3".
A pair ( x , y ) one flashcard : a question and its answer.
Picture: a stack of flashcards; the front is x y
D D the whole pile of flashcards for one task. Written D = {( x i , y i )} i { }
T t T t one job to learn , e.g. "recognise French words." The subscript t T 1 T 2
Picture: a conveyor belt delivering one box (task) at a time — see figure below.Why the topic needs it: ordinary machine learning has one task. Continual learning's whole reason for existing is that tasks arrive one after another in a stream , and the earlier boxes are taken away before the next arrives.
The crucial pain, shown by the crossed-out boxes: once D 1 you cannot look at it again. That is why you can't just "retrain on everything." This single constraint is the seed of the whole field — see Online Learning for the streaming-data setting and Catastrophic Forgetting for what goes wrong.
Definition The model functionf ( x ; θ ) f give it an input x θ The semicolon separates "the thing we feed in" (x θ
Read f ( x ; θ ) "what the machine says about x θ
Why the semicolon and not a comma? It signals θ setting we control , while x data flowing through .
Intuition Why we need a number for wrongnessTo improve, the machine must know how badly it did, as a single number it can try to shrink. A big number = very wrong; zero = perfect. That number is the loss .
Definition Per-example lossℓ L
==ℓ wrongness on ONE flashcard : ℓ ( f ( x ; θ ) , y ) y
==L t t average wrongness over the whole task T t
Picture: ℓ L
Definition The expectationE E ( x , y ) ∼ D [ ⋅ ] "the average value, taken over all flashcards drawn from D The ∼ "drawn from" . So
L t ( θ ) = E ( x , y ) ∼ D t [ ℓ ( f ( x ; θ ) , y ) ] "the task-t f t
Why an average? One card could be lucky or unlucky; the average is the honest overall score.
Here is the tool that makes learning and forgetting happen. It deserves its own picture.
Think of the loss as a hilly landscape : each point on the ground is one knob-setting θ height is the loss L
∇ θ L ∇ θ L an arrow that points in the steepest-uphill direction of the loss landscape , computed with respect to the knobs θ
Why this tool and not another? We want to know which way to turn each knob to reduce loss fastest . The gradient is precisely the answer: it is the multi-knob version of "slope." To go down , we step in the opposite direction, hence the minus sign next.Picture: the red uphill arrow in the figure; we walk against it (green arrow, downhill).
Common mistake The trap hiding in that arrowThe update uses ∇ θ L t only the current task . It knows nothing about old tasks' valleys. Walking downhill for task 2 can walk you straight up the hillside of task 1. That is catastrophic forgetting, and the whole topic exists to fix it.
θ ∗ arg min
==arg min θ L ( θ ) "the knob-setting that gives the smallest loss" — the location of the valley bottom, not the depth.
==θ ∗
Picture: the lowest point of the valley in the previous figure.Why the topic needs it: once we reach θ 1 ∗ ground is flat there — the slope ∇ θ L 1 ( θ 1 ∗ ) ≈ 0
The regularization methods later ask: "how sharply does the loss rise if I move away from the valley bottom?" A gentle valley → moving is cheap; a steep valley → moving is costly.
H H curvature (steepness of the walls) of the loss valley around θ ∗
Picture: a bowl. A narrow deep bowl has high curvature; a wide flat dish has low curvature.
Definition The Fisher InformationF j F j cheap, practical stand-in for the curvature of knob j , i.e. how much the model's predictions change if you jiggle knob j F j j important , don't move it. It approximates the diagonal of H
Why not just use H The full Hessian is a giant table (every knob paired with every other) — too big to compute. F j Links: this curvature idea underpins Regularization Techniques and the EWC method in the parent note.
Definition Remaining symbols in one place
==λ dial for how hard we protect old tasks (stability vs. plasticity trade-off). Big λ
==β how much weight we give replayed old examples during new-task training.
==M memory buffer : a small backpack of saved old flashcards we carry forward.
==p ( y ∣ x ; θ ) probability that the answer is y x ∣ "given."
==σ squashing function that turns any number into a value between 0 and 1 (a probability), e.g. σ ( z ) = 1 + e − z 1
Parameters theta = the knobs
Model f maps input to output
Dataset D = pile of cards
Task loss L = average wrongness
Gradient nabla = steepest uphill arrow
SGD update = step downhill
Best knobs theta-star at valley bottom
Curvature H and Fisher F = wall steepness
Continual Learning methods
The map reads left-to-right into the topic: knobs and data build the loss; the loss builds the gradient and SGD; SGD over a stream of tasks is what causes forgetting; curvature tells us which knobs to protect — and that is where Continual and Lifelong Learning proper begins.
What does the bold x x That the input is a list of numbers (like all pixels of an image), not a single number.
What does θ All the model's knob settings; every task's memory sits in these same numbers, so re-turning them for a new task overwrites old ones.
In words, what does E ( x , y ) ∼ D [ ℓ ( ⋅ )] The average per-example wrongness taken over all flashcards drawn from dataset D
Which direction does ∇ θ L It points steepest uphill ; we step in the opposite (downhill) direction to reduce loss.
Why does ∇ θ L 1 ( θ 1 ∗ ) ≈ 0 At the valley bottom (best task-1 knobs) the loss landscape is flat, so its slope is roughly zero.
What is the difference between arg min min arg min location (which θ min value (how low).
What does a large Fisher value F j j Protect it — knob j
What do the two trade-off dials λ β λ β
Why can't continual learning just retrain on all past data? Because in a task stream , old datasets are removed before new ones arrive, so they are no longer available.