Visual walkthrough — Counterfactual explanations

This is the visual companion to the main note. We only need one setting: an input the model rejected, and the question "what is the smallest nudge that gets it accepted?"

---

Step 1 — What is a "point" and what is the model doing to it?

WHAT. Picture a person applying for a loan described by two numbers: income (horizontal) and credit score (vertical). That pair of numbers is a single point on a flat sheet of paper — a plane. We write it $\mathbf{x}=(x_1,x_2)$. The bold letter just means "a bundle of numbers"; the little subscripts are which slot.

WHY. Before any formula, we must agree that an applicant = a dot on a map. Every idea below is about moving that dot.

PICTURE. The amber dot is our applicant $\mathbf{x}$. The whole map is split into two coloured regions by a line — the model's verdict.

---

Step 2 — The straight-line model: reading off which side you are on

WHAT. The simplest possible border is a straight line. To test which side a dot is on, the model builds one number: $s(\mathbf{x})=\underset{\text{weight on income}}{\underbrace{w_1}}\underset{\text{income}}{\underbrace{x_1}}+\underset{\text{weight on credit}}{\underbrace{w_2}}\underset{\text{credit}}{\underbrace{x_2}}+\underset{\text{offset}}{\underbrace{b}}.$ Then it looks only at the sign of $s$: positive → approved, negative → denied, exactly zero → sitting on the border. We keep this convention for the rest of the page: $s>0$ means approved, $s<0$ means denied.

WHY. We need a rule that turns a 2-D dot into a single "how approved are you" score. A weighted sum is the cheapest such rule. The weights $w_1,w_2$ say how much each feature counts; $b$ shifts the whole border sideways so it need not pass through the origin.

PICTURE. The score $s$ grows as you march in one particular direction. That direction — the direction of steepest increase of the score — is the arrow $\mathbf{w}=(w_1,w_2)$.

---

Step 3 — Why the weight arrow points straight across the fence

WHAT. Take two dots $\mathbf{p}$ and $\mathbf{q}$ that both sit on the boundary. Both have score $0$, so ${\mathbf{w}}^{\top }\mathbf{p}+b=0,{\mathbf{w}}^{\top }\mathbf{q}+b=0.$ Subtract them: ${\mathbf{w}}^{\top }(\mathbf{p}-\mathbf{q})=0$. The little arrow $\mathbf{p}-\mathbf{q}$ runs along the fence.

WHY. A dot product of two arrows equals zero exactly when they are at right angles (this is what "${\mathbf{w}}^{\top }(\cdot )=0$" means geometrically — no overlap in direction). So $\mathbf{w}$ is perpendicular to every direction along the fence. In one sentence: $\mathbf{w}$ is the normal to the boundary — it points straight across it.

PICTURE. The cyan line is the fence; the amber arrow $\mathbf{w}$ juts out of it at a perfect $90^{\circ }$.

---

Step 4 — "Closest point": what are we even minimising?

WHAT. A counterfactual ${\mathbf{x}}^{\prime }$ is the applicant we wish we were: same map, but on the approved side, and as near to us as possible. "Near" needs a ruler. The ruler here is straight-line distance, and minimising distance is the same as minimising its square: $\text{cost}({\mathbf{x}}^{\prime })=\Vert {\mathbf{x}}^{\prime }-\mathbf{x}{\Vert }^2=(x_1^{\prime }-x_1{)}^2+(x_2^{\prime }-x_2{)}^2.$

WHY. Why the square? Because the square has no ugly square-root, so its slope (gradient) is clean, and squaring doesn't change which point is closest — the nearest dot stays nearest. We also add a rule the answer must obey: land on the fence, ${\mathbf{w}}^{\top }{\mathbf{x}}^{\prime }+b=0$. (Reaching the fence is enough to flip the verdict, and going even one inch past it only wastes distance.)

PICTURE. Circles of equal distance grow outward from $\mathbf{x}$. The first circle that kisses the fence touches it at exactly one point — our target.

---

Step 5 — Turning "closest under a rule" into one equation (Lagrange)

WHAT. We minimise distance while obeying the on-the-fence rule. The bookkeeping trick for "minimise A subject to rule B = 0" is to build one combined quantity: $\mathcal{L}({\mathbf{x}}^{\prime },\mu )=\underset{\text{distance to beat}}{\underbrace{\Vert {\mathbf{x}}^{\prime }-\mathbf{x}{\Vert }^2}}+\underset{\text{tug strength}}{\underbrace{\mu }}\underset{\text{the rule, }=0}{\underbrace{({\mathbf{w}}^{\top }{\mathbf{x}}^{\prime }+b)}}.$ The extra knob $\mu$ (a single number, the Lagrange multiplier) tunes how hard the rule pulls.

WHY. At the true best spot, the "make distance smaller" push and the "stay on the fence" push must exactly cancel — otherwise you could slide along the fence and do better. Setting the slope of $\mathcal{L}$ to zero encodes that cancellation in one line.

PICTURE. Two arrows at the answer: the amber "shrink distance" arrow and the cyan "along-the-fence" constraint. They line up perpendicular — the balance condition.

The slope with respect to ${\mathbf{x}}^{\prime }$: ${\nabla }_{{\mathbf{x}}^{\prime }}\mathcal{L}=\underset{\text{from the distance term}}{\underbrace{2({\mathbf{x}}^{\prime }-\mathbf{x})}}+\underset{\text{from the rule}}{\underbrace{\mu \mathbf{w}}}=0.$ Read it directly: the move ${\mathbf{x}}^{\prime }-\mathbf{x}$ must be a multiple of $\mathbf{w}$. Rearranged: ${\mathbf{x}}^{\prime }=\mathbf{x}-\frac{\mu }{2}\mathbf{w}.$ This is the promise from Step 3, now forced by algebra: you move along $\mathbf{w}$, nowhere else.

---

Step 6 — Solving for how far to move, and the final formula

WHAT. We know the direction ($\mathbf{w}$); we need the amount $\mu$. Plug ${\mathbf{x}}^{\prime }=\mathbf{x}-\frac{\mu }{2}\mathbf{w}$ into the rule "land on the fence": ${\mathbf{w}}^{\top }\left(\mathbf{x}-\frac{\mu }{2}\mathbf{w}\right)+b=0.$ Expand: ${\mathbf{w}}^{\top }\mathbf{x}+b-\frac{\mu }{2}\Vert \mathbf{w}{\Vert }^2=0$, so $\frac{\mu }{2}=\frac{{\mathbf{w}}^{\top }\mathbf{x}+b}{\Vert \mathbf{w}{\Vert }^2}.$

WHY. ${\mathbf{w}}^{\top }\mathbf{w}=\Vert \mathbf{w}{\Vert }^2$ is just "the arrow dotted with itself" = its length squared. Substituting the amount back into the direction gives the closed form: $\boxed{{\mathbf{x}}^{\prime }=\mathbf{x}-\frac{{\mathbf{w}}^{\top }\mathbf{x}+b}{\Vert \mathbf{w}{\Vert }^2}\mathbf{w}}$ Term by term: start at $\mathbf{x}$, step against $\mathbf{w}$ (minus sign — you were on the negative side, you climb toward positive), by an amount = how far off the fence your score is $({\mathbf{w}}^{\top }\mathbf{x}+b)$ divided by how strong the arrow is $(\Vert \mathbf{w}{\Vert }^2)$.

PICTURE. The green counterfactual lands exactly on the fence, connected to $\mathbf{x}$ by the shortest possible spoke, parallel to $\mathbf{w}$.

---

Step 7 — The degenerate cases (never let the reader fall off the map)

WHAT & WHY. Three things can go wrong; each has a clear picture.

1. Already approved (${\mathbf{w}}^{\top }\mathbf{x}+b>0$): the step size is positive, so $\mathbf{x}-(\text{positive})\mathbf{w}$ walks you backward onto the fence you already cleared. The formula still works; it just returns a small backward nudge.
2. Exactly on the fence (${\mathbf{w}}^{\top }\mathbf{x}+b=0$): step size $=0$, ${\mathbf{x}}^{\prime }=\mathbf{x}$. You are already the counterfactual — no change needed.
3. Flat model ($\mathbf{w}=0$): then $\Vert \mathbf{w}{\Vert }^2=0$ and the formula divides by zero. Picture: the model gives every dot the same verdict — there is no fence, so no counterfactual exists. The math screaming "divide by zero" is the honest answer "impossible".

PICTURE. Three mini-panels: overshoot-back, zero-move, and no-fence.

---

The one-picture summary

Everything above collapses into a single diagram: dot $\mathbf{x}$, the fence ${\mathbf{w}}^{\top }\mathbf{x}+b=0$, the normal arrow $\mathbf{w}$, and the shortest spoke landing at ${\mathbf{x}}^{\prime }$.

Recall Feynman retelling — say it like a story

Imagine you're a dot on a map, standing in the "denied" field. Somewhere across the map is a straight fence separating "denied" from "approved". You want to reach the approved side by walking the least. Common sense says: walk straight at the fence, not diagonally. But which way is "straight at it"? The model secretly carries an arrow, $\mathbf{w}$, that always points across the fence — because the fence is drawn exactly so that walking along it never changes your score. So you walk toward the approved side along that arrow. How far? Just enough to zero out your score: your current score divided by the arrow's strength tells you the number of steps. Land, and you're on the fence — the smallest possible change that flips the verdict. If the model has no fence (everyone gets the same answer), the arrow is zero, dividing blows up, and honestly there's nothing you can do. That's the whole story, and it's the same story neural nets tell — only their fence bends, so you take baby steps instead of one leap.

Recall Quick self-test

Why must the counterfactual move be parallel to $\mathbf{w}$? ::: Because minimising distance subject to landing on the fence forces $2({\mathbf{x}}^{\prime }-\mathbf{x})+\mu \mathbf{w}=0$, i.e. the move is a multiple of $\mathbf{w}$; geometrically, $\mathbf{w}$ is perpendicular to the fence, the shortest crossing. What does ${\mathbf{w}}^{\top }\mathbf{x}+b$ measure? ::: A signed "how far off the fence" score — positive = approved side, negative = denied side, size = how deep into a region you are. What happens when $\mathbf{w}=0$? ::: The model has no decision boundary; the formula divides by zero — no counterfactual exists. How does this extend to non-linear models? ::: Replace the single global $\mathbf{w}$ by the local gradient $\nabla f$ and take small repeated steps (gradient descent) instead of one jump.

See also: Recourse (turning this move into real-world advice), Causal Inference & FACE-style constraints (which moves are actually allowed), Model Debugging (using counterfactuals to spot spurious fences).