Question bank — Continual and lifelong learning

False. A network with millions of spare parameters still forgets; the cause is that plain SGD reuses the same shared weights and its gradient only knows the current task, so it happily overwrites old configurations. It is an optimisation problem, not a capacity one.

True (locally). If ${\nabla }_{\theta }{\mathcal{L}}_2$ moves $\theta$ only in directions where ${\mathcal{L}}_1$ is flat, task 1's loss barely changes. Forgetting comes precisely from the overlap between the directions each task cares about.

False. A zero gradient means task 2's update starts from ${\theta }^{\ast }$, but the very first step ${\theta }^{\ast }-\eta {\nabla }_{\theta }{\mathcal{L}}_2$ can move straight uphill on ${\mathcal{L}}_1$. Being at a minimum protects you against nothing once you optimise a different objective.

False. EWC stores only ${\theta }_{t-1}^{\ast }$ and the Fisher values $F_j$ — a summary. It never revisits ${\mathcal{D}}_{t-1}$, which is exactly why it is used when old data must be deleted (privacy, storage).

Partly false. Plain replay does need raw data, but generative replay sidesteps this: you train a generator on old tasks and sample synthetic "pseudo-examples", storing no real records.

True by construction. Old columns' parameters are frozen and never updated when a new column is added, so no old prediction can drift. The price is that the model grows with every task.

False. Large $\lambda$ buys stability but kills plasticity — the model becomes so rigid it can barely learn the new task. It is a trade-off dial, not a "bigger is better" knob.

True. If learning a later task actually improves an old task (shared useful features), ${\text{Acc}}_{T,i}>{\text{Acc}}_{i,i}$, giving positive BWT — the good case that meta-learning and shared representations aim for.

False. Reservoir sampling makes every example seen so far equally likely to be in the buffer ($k/N$), regardless of arrival time — so early and late tasks are represented in proportion to how many examples they had, not recency.

Error: you must compare against the task's peak accuracy (the $\max$ over all checkpoints since learning it), not the initial. A task may keep improving for a while before decaying; measuring from initial hides that early gain and understates forgetting.

Error: $F_j$ measures sensitivity of predictions to weight $j$, not the weight's magnitude. A small weight can be highly important (big $F_j$) and a large weight can be irrelevant. Protecting by magnitude is a different, worse heuristic.

Error: it approximates the old loss with a second-order (quadratic/Hessian) Taylor term. The first-order gradient term is dropped because $\nabla {\mathcal{L}}_{t-1}({\theta }_{t-1}^{\ast })\approx 0$ at the minimum; the curvature (Fisher ≈ Hessian diagonal) is what remains.

Error: both gradients are summed (weighted by $\beta$) and the update follows their combination. There is no "winner takes all" — the compromise direction is precisely what enforces stability.

Error: reservoir sampling balances by arrival probability (proportional to task size); class-balanced buffers force equal examples per class. When tasks have skewed class counts these give very different memories, and class-balancing protects rare classes better.

Error: old features $h^{(j)}$ are frozen; the lateral weights $U^{(t\leftarrow j)}$ only learn how to read and reuse them for the new task. The old features themselves never change.

SGD optimises only the current objective with one shared parameter set and no memory of the old loss. Brains use mechanisms like memory consolidation and separate/complementary systems (fast hippocampus, slow cortex) that rehearse and protect old traces — analogous to replay plus regularisation combined.

The full Hessian is huge and expensive, and can have negative eigenvalues (not a valid rigidity measure). The Fisher is a positive, cheap-to-estimate approximation of the Hessian diagonal that only needs first derivatives of the log-likelihood, giving a clean per-weight importance score.

Because the summed loss injects old gradients ${\nabla }_{\theta }{\mathcal{L}}_1,\dots$ back into every update. To lower the total loss, $\theta$ must find a point that is decent for both old and new tasks simultaneously — a compromise no single-task gradient would choose.

The two demands pull the same shared weights in opposite ways: staying put preserves old tasks (stability) but blocks learning new ones (plasticity). With one parameter set you cannot maximise both freely, so every method is really a choice of where on this spectrum to sit.

The generator is itself a continually-learned model, so it forgets old distributions too. Sampling from a decayed generator produces lower-quality pseudo-examples, and errors compound task after task — a "forgetting of the forgetting-preventer".

You can distil the old model's outputs on new (or buffered) inputs as soft targets, penalising the new model for drifting away from old predictions — a data-driven way to preserve old behaviour without keeping old labels.

The penalty term vanishes, so EWC reduces to plain SGD and offers no protection — you get full forgetting. This signals that task 1's predictions were insensitive to all weights, i.e. it was near-trivial for the network.

There is nothing to forget — old and new gradients align, EWC's penalty is automatically satisfied at ${\theta }^{\ast }$, and replay just re-shows the same distribution. This is the degenerate "positive transfer, zero conflict" corner of the problem.

The keep-probability $k/N\to 0$, so any specific old example almost surely gets evicted eventually. Replay's guarantee is statistical coverage of the distribution, never permanence of individual points.

Parameters and lateral connections grow (roughly quadratically in $T$ for the laterals), so memory and compute blow up. Zero forgetting is bought with unbounded model growth — fine for a handful of tasks, impractical for thousands.

The weights are frozen essentially completely at ${\theta }_{t-1}^{\ast }$; plasticity drops to zero and the model cannot learn the new task at all. This is the extreme-stability endpoint of the dilemma.

The old-buffer term disappears and the objective collapses back to pure new-task training — replay in name only, with forgetting returning to the plain-SGD level. $\beta$ is the stability dial for replay just as $\lambda$ is for EWC.

Replay-style and online methods still work because they never needed a boundary — you just keep sampling from the buffer. Boundary-triggered methods like classic EWC (which snapshots ${\theta }^{\ast }$ and computes Fisher at each task end) need adaptation, since there is no clear moment to snapshot.