Activation patching: Worked examples — AI-ML deep dive

This page is the drill ground for Activation patching. The parent note built the machinery: clean input, corrupted input, patch a component, measure how much of the behavioural gap you closed. Here we do something different — we take that one idea and push it into every corner where it behaves strangely, so that when you run a real patching experiment nothing surprises you.

Before we start, three symbols we will lean on constantly.

Now the one number we keep computing — the parent's causal fraction:

That last sentence already tells you this number is not automatically trapped in $[0, 1]$ . The parent note said "Fraction ∈ [0,1]" as the happy case — this page is where we meet the unhappy cases.

The scenario matrix

Every patching experiment lands in one of these cells. The examples below hit each one at least once.

#	Cell (the "case class")	What's weird about it	Example
A	Clean textbook case — patch fully restores	$CF \approx 1$ , the ideal	Ex 1
B	Sign flip: negative CF — patch hurts	numerator negative	Ex 2
C	Overshoot: CF > 1 — patched beats clean	patched run better than clean	Ex 3
D	Zero / degenerate denominator — clean = corrupt	you can't divide; formula undefined	Ex 4
E	Partial + hierarchy — need multiple components	one alone gives fraction, sum gives whole	Ex 5
F	Redundancy: negative synergy — two heads overlap	$Δ_{A \cup B} < Δ_{A} + Δ_{B}$	Ex 6
G	Limiting behaviour — patch the whole model vs nothing	boundary values $CF = 1$ and $CF = 0$ forced	Ex 7
H	Wrong-metric trap — same patch, two metrics disagree	metric choice changes the conclusion	Ex 8
I	Real-world word problem — end to end	translate a story into $L$ values	Ex 9
J	Exam twist — direction of patching reversed	patch clean→corrupt (noising) not corrupt→clean	Ex 10

Figure s01 — the CF map. A horizontal number line for $CF$ . Tick marks sit at $0$ and $1$ . The strip left of 0 is shaded red ("patch broke it"), the strip between 0 and 1 is shaded green ("partial / full fix"), and the strip right of 1 is shaded yellow ("overshoot"). Coloured dots mark where five of our examples land: Ex 2 at $- 0.163$ (red), Ex 7a at $0$ , Ex 1 at $0.898$ (green), Ex 7b at $1$ , and Ex 3 at $1.041$ (yellow). Keep glancing back — every example is a dot somewhere on this line.

Cell A — the clean textbook case

Worked example Ex 1 · IOI head 9.6 (cell A)

Same IOI setup as the parent note. Losses (golf scores, lower = better):

$L (y_{corrupt}) = 5.2$
$L (y_{patched}) = 0.8$
$L (y_{clean}) = 0.3$

Forecast: guess before computing — is this patch worth almost nothing, about half, or almost everything?

Step 1. Compute the total gap (denominator): $5.2 - 0.3 = 4.9$ . Why this step? This is the full distance from broken to correct. Everything is measured relative to it.

Step 2. Compute the distance travelled (numerator): $5.2 - 0.8 = 4.4$ . Why this step? Patching dropped the loss from $5.2$ to $0.8$ ; that drop is our progress.

Step 3. Divide: $CF = 4.4/4.9 \approx 0.898$ . Why this step? The fraction is the whole point — it says head 9.6 explains ~90% of the gap.

Verify: $CF \in (0, 1)$ ✓ (patched loss $0.8$ sits between clean $0.3$ and corrupt $5.2$ , so the fraction must land in $(0, 1)$ — a free sanity check). Sits on the green dot in the figure. ✓

Cell B — the patch makes things worse

Worked example Ex 2 · A distractor head (cell B)

You patch head 3.2 (a wrong guess). Measurements:

$L (y_{corrupt}) = 5.2$ , $L (y_{clean}) = 0.3$ (same task as Ex 1)
$L (y_{patched}) = 6.0$

Forecast: the patched loss ( $6.0$ ) is higher than the corrupted loss ( $5.2$ ). What sign will $CF$ have?

Step 1. Numerator: $5.2 - 6.0 = - 0.8$ . Why this step? A negative numerator means the loss went up after patching — the model got worse.

Step 2. Denominator: still $5.2 - 0.3 = 4.9$ (positive, task unchanged). Why this step? The gap is a property of the task, not of which component you poked.

Step 3. $CF = - 0.8/4.9 \approx - 0.163$ . Why this step? A negative causal fraction. This head, when fed its clean activation, actively disrupts the corrupted run — perhaps it carries information that only makes sense in the clean context.

Verify: $CF < 0$ ✓ — matches "patched loss > corrupt loss" (red region in the figure). Do not conclude "this head is important"; conclude "patching it here is counterproductive." ✗-lesson.

Cell C — patched beats clean (overshoot)

Worked example Ex 3 · Over-restoration (cell C)

A layer whose clean activation is a very strong signal. Measurements:

$L (y_{corrupt}) = 5.2$ , $L (y_{clean}) = 0.3$
$L (y_{patched}) = 0.1$

Forecast: patched loss $0.1$ is below the clean loss $0.3$ . Where does $CF$ land relative to $1$ ?

Step 1. Numerator: $5.2 - 0.1 = 5.1$ . Why this step? This is the distance travelled — patching dropped the loss from $5.2$ all the way to $0.1$ , so the "progress" is a big number.

Step 2. Denominator: $5.2 - 0.3 = 4.9$ . Why this step? This is the full clean-to-corrupt gap, our reference span; notice the numerator ( $5.1$ ) is larger than it, which already forecasts $CF > 1$ .

Step 3. $CF = 5.1/4.9 \approx 1.041 > 1$ . Why this step? The corrupted run plus this clean patch answers even more confidently than the fully clean run. That happens when the corrupted prompt still supplies some helpful context that the clean prompt lacked, and the patch adds the missing piece on top.

Verify: $CF > 1$ ✓ (patched $0.1 <$ clean $0.3$ ). Yellow "overshoot" region. Interpretation: the component is more than sufficient — a valid, real, and often-overlooked outcome.

Cell D — the degenerate case (you cannot divide)

Worked example Ex 4 · Clean and corrupt behave identically (cell D)

You pick a bad corrupted prompt. It turns out the model answers correctly on it anyway.

$L (y_{corrupt}) = 0.30$
$L (y_{clean}) = 0.30$
$L (y_{patched}) = 0.29$

Forecast: what is the denominator? What does that do to the fraction?

Step 1. Denominator: $0.30 - 0.30 = 0$ . Why this step? There is no gap to close — clean and corrupted are the same behaviour.

Step 2. The formula $CF = (0.30 - 0.29) /0$ is $0.01/0$ = undefined (division by zero). Why this step? You cannot express "fraction of a distance" when the distance is zero. Any nonzero numerator explodes; a zero numerator is $0/0$ .

Step 3. Diagnosis, not computation. The fix is to choose a corrupted input that actually breaks the model so the denominator is safely positive (say $> 1$ nat). This is a design requirement of the experiment, not a maths bug. Why this step? Patching only measures something when there is a behavioural difference to attribute.

Verify: denominator $= 0$ ✓, so $CF$ is undefined ✓. This is the single most common wasted experiment — always print the raw gap before dividing. This is exactly the kind of guard Causal Scrubbing formalises: your corruption must destroy the behaviour you're testing.

Cell E — partial effects that add up (hierarchy)

Worked example Ex 5 · Greater-than, L15 + L20 (cell E)

Task (self-contained): the model completes "The war lasted from 1712 to 17__" and should predict a two-digit year greater than 12 (like "15" or "83"), never a year $\leq 12$ . We define a metric called valid-mass = the total probability the model puts on all years greater than 12. Higher valid-mass = more correct behaviour.

Turn that into a loss so lower = better: $L = 1 - (valid-mass)$ . Four runs, each giving a valid-mass we convert:

Corrupt (a broken prompt that anchors the model on "99"): valid-mass $0.20 \Rightarrow L_{corrupt} = 1 - 0.20 = 0.80$
Clean (the good prompt anchored on "12"): valid-mass $0.95 \Rightarrow L_{clean} = 1 - 0.95 = 0.05$
Patch L15 only (layer 15 does the numerical "greater-than" reasoning): valid-mass $0.75 \Rightarrow L = 0.25$
Patch both L15 + L20 (layer 20 converts the internal number into output tokens): valid-mass $0.93 \Rightarrow L = 0.07$

Forecast: will L15 alone close most, half, or little of the gap?

Step 1. Total gap (denominator): $L_{corrupt} - L_{clean} = 0.80 - 0.05 = 0.75$ . Why this step? This is the full distance from broken to correct in valid-mass terms — every fraction below is measured against it, so we compute it once first. Step 2. L15 fraction: $(0.80 - 0.25) /0.75 = 0.55/0.75 \approx 0.733$ . Why this step? L15 does the numerical reasoning; alone it recovers ~73% of the behaviour. Step 3. Both fraction: $(0.80 - 0.07) /0.75 = 0.73/0.75 \approx 0.973$ . Why this step? Adding L20 (the token-formatting layer) pushes us to ~97% — the full causal chain.

Verify: both-fraction ( $0.973$ ) > L15-fraction ( $0.733$ ) ✓, and both stay in $(0, 1)$ ✓. Matches the story "L15 reasons, L20 formats" — see Causal Mediation Analysis for the formal mediator framing.

Cell F — redundancy shows up as negative synergy

Worked example Ex 6 · Two duplicate name-mover heads (cell F)

Two heads A and B each partly recover the IOI answer. First, the symbol we need:

$Δ$ = the raw loss drop from patching a component, $Δ = L (y_{corrupt}) - L (y_{patched})$ . It is the numerator of CF (the unnormalised distance travelled), measured in nats here. Bigger $Δ$ = the patch helped more.

Their individual effects:

$Δ_{A} = 3.0$ , $Δ_{B} = 2.8$
Patch both together: $Δ_{A \cup B} = 3.2$

Forecast: is $3.2$ bigger or smaller than $3.0 + 2.8$ ? What sign is the synergy?

Step 1. Recall the parent's synergy formula: $Synergy (A, B) = Δ_{A \cup B} - (Δ_{A} + Δ_{B})$ . Why this step? It compares "effect of the team" to "sum of solo effects." Step 2. Plug in: $3.2 - (3.0 + 2.8) = 3.2 - 5.8 = - 2.6$ . Why this step? Strongly negative → the two heads are redundant, both computing the same "Mary is the object" signal. Step 3. Interpretation: patching them one at a time each looked ~fully effective; patching together adds almost nothing new. This is exactly the "patch everything at once" trap from the parent note, now with numbers.

Verify: synergy $= - 2.6 < 0$ ✓. Compare with Ablation Studies: ablating one redundant head barely hurts, which under-states importance — the mirror image of this over-statement.

Cell G — the two limiting endpoints

Worked example Ex 7 · Boundary sanity checks (cell G)

Two trivial but essential limits, same task with $L_{corrupt} = 5.2$ , $L_{clean} = 0.3$ .

(a) Patch nothing. Then $y_{patched} = y_{corrupt}$ , so $L_{patched} = 5.2$ . Step 1. $CF = (5.2 - 5.2) / (5.2 - 0.3) = 0/4.9 = 0$ . Why this step? Zero components patched must recover zero of the gap — a hard boundary the formula must respect, or it's wrong.

(b) Patch the entire network (every activation). Then the corrupted run is byte-for-byte the clean run: $L_{patched} = 0.3$ . Step 2. $CF = (5.2 - 0.3) / (5.2 - 0.3) = 4.9/4.9 = 1$ . Why this step? Patching everything is identical to running clean, so it must give exactly $1$ .

Verify: the two extremes give exactly $0$ and $1$ ✓. Any real single-component result lives between these unless it overshoots (Ex 3) or backfires (Ex 2). These endpoints are your unit test: run them first; if they don't return $0$ and $1$ , your patching harness is buggy — the same "does the full patch = clean run?" check the Logit Lens tooling relies on.

Cell H — the metric decides the conclusion

Worked example Ex 8 · Same patch, two metrics disagree (cell H)

Toxicity study. You patch layer 12. Two metrics on the same patched run:

Cross-entropy over all tokens (averaged next-token loss, in nats): corrupt $2.00$ , patched $1.98$ , clean $1.95$ .
Targeted logit difference $(logit(safe) - logit(toxic))$ turned into a loss $L = - (logit diff)$ so lower = safer: corrupt $2.0$ , patched $- 1.5$ , clean $- 2.0$ .

Forecast: one metric will say "barely mattered," the other "hugely mattered." Which is which?

Step 1. Cross-entropy gap (denominator): $2.00 - 1.95 = 0.05$ ; numerator $2.00 - 1.98 = 0.02$ ; so $CF_{CE} = 0.02/0.05 = 0.4$ . Why this step? Averaged over all tokens, the toxic token is one drop in an ocean — the effect looks middling. Step 2. Targeted gap (denominator): $2.0 - (- 2.0) = 4.0$ ; numerator $2.0 - (- 1.5) = 3.5$ ; so $CF_{tgt} = 3.5/4.0 = 0.875$ . Why this step? This metric only watches safe-vs-toxic, so the same intervention now looks decisive (~88%). Negative losses are fine — the subtraction still measures distance. Step 3. Conclusion: the metric, not the model, changed the story. For a toxicity question the targeted logit difference is the honest metric (parent note, Mistake 3).

Verify: $0.4 \neq = 0.875$ ✓ — a single patch legitimately yields two very different fractions. Always report which metric.

Cell I — a real-world word problem, end to end

Worked example Ex 9 · Translating a story into numbers (cell I)

A model completes "The Eiffel Tower is located in the city of ___" with Paris. You suspect a mid-layer "country/landmark" head. You build:

Clean: "The Eiffel Tower is located in the city of" → $P (Paris) = 0.90$ , so $L (y_{clean}) = - ln 0.90 = 0.105$ .
Corrupt: "The Colosseum is located in the city of" → the model, run with the corrupted subject, gives $P (Paris) = 0.02$ , so $L (y_{corrupt}) = - ln 0.02 = 3.912$ .
Patch the suspect head's clean activation into the corrupt run → $P (Paris) = 0.55$ , so $L (y_{patched}) = - ln 0.55 = 0.598$ .

(Metric = negative log-probability of the clean answer "Paris", in nats; lower = more Paris.)

Forecast: partway, or nearly all the way?

Step 1. Convert each probability to a loss with $L = - ln P$ . Why this step? Probabilities aren't distances; the log turns "how confident in Paris" into a loss we can subtract. We use $ln$ (natural log) so the unit is nats. Step 2. Gap (denominator): $3.912 - 0.105 = 3.807$ nats. Why this step? This is the full "distance from Colosseum-confusion back to Eiffel-Paris." Step 3. Numerator: $3.912 - 0.598 = 3.314$ nats. Why this step? Patching the head dropped the loss from $3.912$ to $0.598$ — that drop is our progress. Step 4. Divide: $CF = 3.314/3.807 \approx 0.870$ . Why this step? The head restores ~87% of the "Paris-ness" — strong evidence it carries the Eiffel→Paris association.

Verify: recompute $- ln 0.90 = 0.1054$ , $- ln 0.02 = 3.912$ , $- ln 0.55 = 0.5978$ ✓; final CF $\approx 0.870 \in (0, 1)$ ✓. Units are nats throughout — consistent. This is the same recipe Mechanistic Interpretability uses to localise factual recall.

Cell J — the exam twist: patch the other direction

Worked example Ex 10 · Noising instead of denoising (cell J)

Twist: so far we patched clean → corrupt ("denoising": can I fix the broken run?). The exam asks the reverse — patch corrupt → clean ("noising": can I break the working run by inserting one corrupted activation?).

Setup — the clean run is the baseline we now try to sabotage:

$L (y_{clean}) = 0.30$ (working)
$L (y_{corrupt}) = 5.20$ (broken, the source of the bad activation)
Insert head 9.6's corrupted activation into the clean run → call this loss $L_{noised} = 4.50$ .

Forecast: if this head is causal, does injecting its corrupt value push the clean run toward broken?

Step 1. For noising, flip the reference frame: measure the fraction of the way from clean toward corrupt. $CF_{noise} = \frac{L _{noised} - L ( y _{clean} )}{L ( y _{corrupt} ) - L ( y _{clean} )}$ Why this step? The "start" is now the clean loss and the "target" is the corrupt loss, so the numerator measures how far we drifted away from correct. Step 2. Numerator: $4.50 - 0.30 = 4.20$ . Why this step? This is how far the injected corrupt activation dragged the clean run away from its correct baseline. Step 3. Denominator: $5.20 - 0.30 = 4.90$ . Why this step? The full distance from clean all the way to fully broken — our reference span. Step 4. Divide: $CF_{noise} = 4.20/4.90 \approx 0.857$ . Why this step? Injecting just this head's corrupt value breaks ~86% of the correct behaviour — a second, independent confirmation that head 9.6 is load-bearing.

Verify: $CF_{noise} \approx 0.857 \in (0, 1)$ ✓. Denoising (Ex 1, $0.898$ ) and noising ( $0.857$ ) both pointing high is the gold standard — a head that both fixes the broken run and breaks the working run is genuinely causal, not a fluke of one direction.

Recall Self-test (reveal after answering)

What does CF = 0 mean? ::: The patch closed none of the gap — patched loss equals corrupt loss (patching nothing, Ex 7a). What does CF > 1 mean? ::: Overshoot — the patched run is better than the fully clean run (Ex 3). Why can CF be undefined? ::: Clean and corrupt behave identically → denominator is zero → no gap to attribute (Ex 4). A head gives high CF for denoising but ~0 for noising — trustworthy? ::: Weakly; the gold standard needs both directions high (Ex 10). Two heads: $Δ_{A} = 3, Δ_{B} = 2.8, Δ_{A \cup B} = 3.2$ . Redundant or synergistic? ::: Redundant — synergy $= - 2.6 < 0$ (Ex 6).