Exercises — Activation patching

It measures causation: whether forcing a component to carry clean information changes the output. Attention weights only show what a head looks at (correlation), not whether the model uses it.

Patched loss equals corrupt loss, so the patch changed nothing. $\text{CF}=\frac{4.0-4.0}{4.0-0.5}=\frac{0}{3.5}=0$ The component is not causally important for this behaviour.

Negative ${\Delta }_C$ means $\mathcal{L}(y_{\text{patched}})>\mathcal{L}(y_{\text{corrupt}})$ — patching made things worse. Copying that clean activation hurt the corrupted run, e.g. the component's clean value clashes with the rest of the corrupted context.

$\text{CF}=\frac{5.2-0.8}{5.2-0.3}=\frac{4.4}{4.9}=0.898\dots \approx 0.90$ The head explains ~90% of the corrupted error.

$\text{CF}=\frac{5.2-3.7}{5.2-0.3}=\frac{1.5}{4.9}=0.306\approx 0.31$ Only ~31% of the gap closed — much less important than head 9.6's 90%.

Here the metric goes the other way — bigger mass is better — so the "fraction restored" uses mass directly: $\text{effect}=\frac{{\text{mass}}_{\text{patched}}-{\text{mass}}_{\text{corrupt}}}{{\text{mass}}_{\text{clean}}-{\text{mass}}_{\text{corrupt}}}=\frac{0.75-0.20}{1.00-0.20}=\frac{0.55}{0.80}=0.6875\approx 0.69$ This restores ~0.69 of the missing greater-than mass. (The parent's table lists $0.65$ as a raw effect scale; our normalized fraction is the more interpretable form — both agree on "most of the behaviour returns".)

$\text{Synergy}=0.65-(0.10+0.10)=0.45>0$ Positive synergy → the heads collaborate: neither does much alone, but together they carry the behaviour. This is why single-head patching under-reports their importance.

$\text{Synergy}=0.55-(0.50+0.50)=-0.45<0$ Negative → redundancy: both heads compute the same thing, so patching one already recovers most of the effect and adding the second barely helps. Related to Ablation Studies, where removing one redundant unit shows little damage.

Swing $=(-1.5)-(+3.0)=-4.5$ logits — a large, decisive change. Trust the targeted logit difference. Cross-entropy averages over the whole vocabulary and drowns the one comparison you care about (toxic vs safe). The metric must measure the capability under study, not generic next-token accuracy.

$\text{Synergy}(L15,L20)=0.92-(0.65+0.05)=0.92-0.70=0.22>0$ Positive synergy confirms a hierarchical pipeline: L15 does the numerical "greater-than" reasoning, L20 formats the number into a token. Neither is sufficient alone — the reasoning must happen and be turned into output tokens. This is the causal chain Logit Lens visualizes layer-by-layer, and Causal Mediation Analysis formalizes as a mediator sequence.

• Clean: "The keys to the cabinet ___" → correct answer "are" (plural, agrees with keys).
• Corrupted: "The key to the cabinet ___" → correct answer "is" (singular). Only the subject number changed, everything else identical — this keeps the intervention surgical.
• Metric: targeted logit difference $\text{logit(are)}-\text{logit(is)}$.
• Procedure: patch each head's activation from the plural-clean run into the singular-corrupt run; the head whose patch flips the logit difference toward "are" is carrying subject number. This is the Mechanistic Interpretability recipe of minimal-pair inputs.

The full-output patch changes what $A$ sends to every downstream component; the path patch changes only what reaches $B$. So a non-zero path effect proves the mechanism specifically routes $A\to B$, ruling out "$A$ mattered through some other head."

$\text{CF}=\frac{4.0-0.1}{4.0-0.5}=\frac{3.9}{3.5}=1.114\dots \approx 1.11$ Not a bug, but a flag: the patched run is more confident than the clean run itself. This can happen when the clean input is itself slightly hard; it means the component, given clean context, over-drives the correct answer. Report it honestly — it signals the corrupted context helped in some incidental way, or the metric is unstable.

Denominator $=0.5-0.5=0$, so $\text{CF}$ is undefined (division by zero). Conclusion: there is no behavioural gap to explain — the corruption didn't actually break the model, so patching is meaningless. Fix the experiment: find a corrupted input that genuinely lowers performance before measuring any component.

(a) $\text{Synergy}=0.80-(0.30+0.10+0.05)=0.80-0.45=0.35$ (b) Distributed. No single layer reaches even half the joint effect ($0.30$ is the max), yet together they hit $0.80$ with large positive synergy — the capability is spread across cooperating layers, not localized. Localized would look like one layer near $0.80$ and tiny synergy.

Causal fraction with loss metric ::: $({\mathcal{L}}_{\text{corrupt}}-{\mathcal{L}}_{\text{patched}})/({\mathcal{L}}_{\text{corrupt}}-{\mathcal{L}}_{\text{clean}})$ Causal fraction with a "good = large" mass metric ::: $({\text{mass}}_{\text{patched}}-{\text{mass}}_{\text{corrupt}})/({\text{mass}}_{\text{clean}}-{\text{mass}}_{\text{corrupt}})$ Synergy of two components ::: ${\Delta }_{A\cup B}-({\Delta }_A+{\Delta }_B)$ Positive synergy means ::: components collaborate (each weak alone) Negative synergy means ::: components are redundant (compute the same thing) Undefined causal fraction means ::: no behavioural gap — the corruption didn't break the model Why a minimal pair matters ::: it isolates one feature so the patch effect is interpretable