Catastrophic forgetting

WHY does this happen? (First principles)

A neural network stores everything it knows in one shared set of weights $\theta$. There is no separate "memory slot" per task — knowledge is distributed and overlapping.

When you fine-tune on a new dataset $D_{\text{new}}$, you minimize a loss that only mentions the new data:

$\mathcal{L}_{\text{new}}(\theta) = \frac{1}{|D_{\text{new}}|}\sum_{(x,y)\in D_{\text{new}}} \ell(f_\theta(x), y)$

Gradient descent takes steps:

$\theta \leftarrow \theta - \eta \,\nabla_\theta \mathcal{L}_{\text{new}}(\theta)$

Nothing in this update references the old task's loss $\mathcal{L}_{\text{old}}$. So the optimizer happily moves $\theta$ out of the region that was good for the old task, as long as it improves the new task. The old skill is collateral damage.

WHAT does it look like?

Imagine two loss landscapes over the same weight vector $\theta$:

• $\mathcal{L}_{\text{old}}(\theta)$ — a valley whose bottom is the "good at task A" region.
• $\mathcal{L}_{\text{new}}(\theta)$ — a different valley for task B.

Their minima are usually in different places. Pure fine-tuning slides $\theta$ into the new valley, climbing out of the old one.

HOW do we derive a fix? (EWC from scratch)

Goal: train on the new task without leaving the old task's good region. We want $\theta$ that is good on new and stays near the old optimum $\theta^*$, but only "stiff" along directions that mattered for the old task.

Step 1 — Bayesian view. After learning the old task we have a posterior $p(\theta \mid D_{\text{old}})$. When new data arrives, Bayes says:

$\log p(\theta \mid D_{\text{old}}, D_{\text{new}}) = \log p(D_{\text{new}}\mid\theta) + \log p(\theta\mid D_{\text{old}}) - \log p(D_{\text{new}})$

Why this step? The old data's influence is fully captured by the posterior $\log p(\theta\mid D_{\text{old}})$ — we don't need to keep the old data around, just remember what it told us about $\theta$.

Step 2 — Approximate the old posterior as a Gaussian (Laplace approximation) centered at $\theta^*$ with precision given by the Fisher information $F$:

$\log p(\theta\mid D_{\text{old}}) \approx -\tfrac{1}{2}\sum_i F_i\,(\theta_i - \theta_i^*)^2 + \text{const}$

Why this step? Near a minimum any smooth loss looks quadratic; the curvature (2nd derivative) tells us how important each weight was. $F_i$ large ⇒ weight $i$ mattered a lot ⇒ don't move it.

Step 3 — Combine into a trainable loss (Elastic Weight Consolidation):

$\boxed{\;\mathcal{L}_{\text{EWC}}(\theta) = \mathcal{L}_{\text{new}}(\theta) + \frac{\lambda}{2}\sum_i F_i\,(\theta_i - \theta_i^*)^2\;}$

Other mitigation strategies (80/20)

The 20% you must know:

Steel-manned mistakes

Connections

• Fine-Tuning LLMs — where forgetting bites hardest.
• LoRA and Adapters — parameter-isolation fix.
• Fisher Information — the "importance" $F_i$ in EWC.
• Laplace Approximation — how the Gaussian posterior arises.
• Continual Learning — the broader problem setting.
• Multi-task Learning — the "train everything jointly" ideal EWC/rehearsal approximate.
• Learning Rate & Optimization — smaller LR reduces drift.

Flashcards

Concept Map

Hinglish (regional understanding)