Counterfactual explanations: Foundations — AI-ML deep dive

This is the prerequisite page for Counterfactual explanations (6.3.10). The parent note throws a lot of notation at you very fast — vectors, norms, gradients, decision boundaries, Lagrange multipliers. Here we build each symbol from nothing, in the order they depend on each other, with a picture for every one.

0. The scene: a model that says yes or no

Before any symbol, picture the situation. A bank has a machine. You feed it facts about a person; it prints APPROVED or DENIED.

Everything below is just careful notation for "the input", "the machine", and "the answer".

1. The vector $x$ — a bundle of numbers with an address

The parent writes $x \in R^{d}$ . Let's earn every piece.

So for our loan applicant:

$x = (income, x_{1} 45, credit, x_{2} 680, years, x_{3} 2)$

What the figure shows: two axes — income (feature $x_{1}$ ) horizontal, credit score (feature $x_{2}$ ) vertical. A black dot sits at $(45, 680)$ labelled $x$ , with dashed guide lines dropping to each axis so you can read its two coordinates. A red dot at $(52, 680)$ labelled $x^{'}$ sits directly to the right at the same height — same credit, higher income. The picture makes the point literal: a person is a single dot, and editing one feature slides the dot along one axis.

Why the topic needs this. A counterfactual is a second point $x^{'}$ near the first. To say "near" you first need "point". No vector, no notion of distance, no counterfactual.

2. The prime mark $x^{'}$ — "the edited version"

$x = (45, 680, 2) x^{'} = (52, 680, 2)$

Here only slot 1 changed ( $45 \to 52$ ). The prime never means multiply or derivative in this topic — it always means "the candidate counterfactual".

Why the topic needs this. The whole page is a hunt for the best $x^{'}$ . We need a name that says "not the original, the proposed edit" without inventing a new letter each time.

3. The function $f$ — the machine as a symbol

Decode that arrow notation:

Sometimes $f$ outputs not a hard $0/1$ but a probability $p$ between $0$ and $1$ (the parent's "probability 0.35"). Read $p$ as the machine's confidence that the answer is 1. The rule: if $p > 0.5$ call it class 1, else class 0.

Definition The two output conventions (

$0/1$ vs $\pm$ ) and how they line up Two labelling styles appear in this topic and they must be made to agree:

${0, 1}$ style (used everywhere above): $0$ = deny, $1$ = approve.
$\pm$ (sign) style: some models first compute a score — a single real number — and then take its sign. We call that score $s$ . The operation $sign (s)$ returns $+ 1$ if the score is positive and $- 1$ if it is negative.

(Exactly how a linear model produces the score $s$ — using weights and a bias — is built later in §6; here we only need to know that $s$ is "one number summarising the applicant" and its sign picks the side of the fence.)

The bridge between the two conventions is a simple relabelling: $sign result + 1 ⟷ class 1 (approve), sign result - 1 ⟷ class 0 (deny) .$ Concretely, if a model outputs the score $s$ , define $f (x) = \frac{sign ( s ) + 1}{2}$ so $+ 1 \mapsto 1$ and $- 1 \mapsto 0$ . Same decision, just two names for the two sides of the fence.

Why the topic needs this. A counterfactual is defined by $f (x^{'}) \neq = f (x)$ — the machine's output must flip. We can't state "flip the answer" without a name for "the answer": that name is $f$ .

4. Distance and norms — how far apart two points are

We want $x^{'}$ close to $x$ . "Close" needs a number.

Now we squeeze that list of changes into a single "how big was the edit" number. Two common ways:

What the figure shows: a black start dot at the origin (labelled "start $x$ ") and a black target dot up-and-right at $(4, 3)$ (labelled "target $x^{'}$ "). The horizontal axis is "change in feature 1", the vertical axis "change in feature 2". A black diagonal straight line connects them — the $L_{2}$ route that alters both features at once. A red two-segment path goes fully right along the bottom then straight up — the $L_{1}$ (city-block) route that spends its whole budget on one feature before touching the other. This is the visual argument for why $L_{1}$ produces sparse edits.

The figure shows why the parent prefers $L_{1}$ for tabular data. To reach a diagonal target, $L_{1}$ (city-block, red path) is happiest going straight along one axis — it likes changing one feature a lot and leaving others at zero. $L_{2}$ (diagonal, black) spreads the change across several features. For a loan you want a clean story ("raise income by $7k") not "nudge all six features a little", so $L_{1}$ wins.

Why the topic needs this. "Minimal change" is meaningless until you pick a ruler. The norm is that ruler.

4b. Categorical and discrete features — the edge case

Not every feature is a smooth number. "Employment type" is one of {salaried, self-employed, unemployed}; "has a degree" is yes/no. You cannot ask an optimizer to "add 0.3 of a category". Here is how the topic handles it.

Why the topic needs this. Loan, hiring, and medical data are mostly tabular with many categorical columns. A counterfactual method that only knows continuous numbers would produce nonsense edits like "become 0.4 self-employed".

5. Argmin, min, and $λ$ — the language of "find the best edit"

The parent's central formula is an $ar g min$ . Let's build it.

Now the "did it flip?" term needs a precise form. There are two honest ways to write it:

So the parent's objective reads, in English:

$x^{'} = ar g min_{z \in X_{valid}} [stay close ∥ z - x ∥_{1} + λ \cdot turn the answer hinge (f (z)) + γ \cdot stay realistic P (z)]$

"Among all valid candidate points $z$ , hand me the one that is closest, flips the answer, and stays realistic — traded off by the dials $λ, γ$ ." (The plausibility term $P$ is defined precisely in §8.)

Why the topic needs this. Three goals fight each other. $ar g min$ + weighted loss is exactly the grammar for "satisfy competing goals, best compromise wins".

6. The decision boundary — where the answer flips

For a simple linear model the parent writes $f (x) = sign (w^{⊤} x + b)$ (recall from §3 that $+ 1$ means approve, $- 1$ means deny, and the thing inside $sign$ is the score $s = w^{⊤} x + b$ ). Decode:

The fence itself is the set of points where the score is exactly zero: $w^{⊤} x + b = 0$ .

What the figure shows: a black straight line cutting diagonally across the plane — the decision boundary $w^{⊤} x + b = 0$ . The upper-left region is labelled "APPROVE side", the lower-right "DENY side". A black dot $x$ sits in the DENY region. A red arrow leaves $x$ and strikes the boundary at a right angle (a small square marks the $90°$ corner), landing on the red target point $x_{boundary}$ — the nearest place the answer flips. The arrow is perpendicular to the fence, which is the visual heart of the derivation below.

Here is the geometric reasoning, step by step, for the parent's closest-boundary formula.

Intuition Why the shortest hop is

perpendicular, and where the formula comes from What we want: the point $x_{boundary}$ on the fence nearest to $x$ . Key geometric fact: among all straight paths from a point to a line, the shortest is the one that hits the line at a right angle. Any slanted path is the hypotenuse of a right triangle whose vertical leg is the perpendicular drop — and a hypotenuse is always longer than a leg. So the shortest hop points straight into the fence. Which direction is "straight into the fence"? The weight vector $w$ points perpendicular to the fence (because the fence is exactly where $w^{⊤} x + b$ stops changing along it, and $w$ is the direction of fastest change). So we must move from $x$ along $w$ , i.e. $x_{boundary} = x - t w$ for some step length $t$ . How far? (mini-derivation of the score-change rate.) Move from $x$ to $x - t w$ and recompute the score: $w^{⊤} (x - t w) + b = (w^{⊤} x + b) - t (w^{⊤} w) .$ The new score is the old score minus $t$ times $w^{⊤} w$ . And $w^{⊤} w = w_{1}^{2} + w_{2}^{2} + \dots = ∥ w ∥^{2}$ (a vector dotted with itself is its squared length). So each unit of $t$ lowers the score by exactly $∥ w ∥^{2}$ . Set the new score to $0$ and solve: $t = \frac{w ^{⊤} x + b}{∥ w ∥ ^{2}}$ .

Putting that step length back in gives the parent's formula:

$x_{boundary} = x - \frac{w ^{⊤} x + b}{∥ w ∥ ^{2}} w .$

That red landing point is the counterfactual for a linear model — the nearest place the answer changes.

Why the topic needs this. The geometry section of the parent is entirely "how do I reach the fence with the shortest hop?". Boundary + perpendicular arrow is the picture behind every formula there.

7. The gradient $\nabla$ — the compass pointing uphill

For real (nonlinear) models there's no clean formula; we walk to the boundary. The tool for walking is the gradient.

Why the topic needs this. "Gradient-based (Wachter et al.)" is the workhorse method. It literally means: start at $x$ , repeatedly step downhill on $L$ until the prediction flips. $\nabla$ , the minus, and $α$ are the three pieces of that step.

8. Plausibility and constraints — staying on planet Earth

We must actually pick a formula for $P$ . Three concrete, commonly-used choices:

Formula Three ways to compute

$P (x^{'})$ 1. Nearest-neighbour distance — how far the candidate is from any real training point: $P_{NN} (x^{'}) = min_{x_{i} \in train} ∥ x^{'} - x_{i} ∥_{2} .$ Far from every real person $\Rightarrow$ big $P$ $\Rightarrow$ penalised. (Here $x_{i}$ ranges over the stored training examples.)

2. Autoencoder reconstruction error — train a network $g$ that learns to compress-then-rebuild realistic inputs. Realistic points rebuild almost perfectly; weird points rebuild badly: $P_{AE} (x^{'}) = ∥ g (x^{'}) - x^{'} ∥_{2}^{2} .$ Big rebuild error $\Rightarrow$ off the manifold $\Rightarrow$ penalised.

3. MAD-weighted spread — divide each feature's change by that feature's median absolute deviation $MAD_{i}$ (a robust "typical spread" of feature $i$ in the data), so a change is only cheap if it is small relative to how much that feature normally varies: $P_{MAD} (x^{'}) = \sum_{i = 1}^{d} \frac{∣ x _{i}^{'} - x _{i} ∣}{MAD _{i}} .$

How the foundations feed the topic

Read top to bottom: points and edits give you distance; categoricals fit into the same space; the model gives you the boundary; loss packages the three goals; the gradient walks you across; plausibility keeps it real. Their meeting point is the counterfactual.

Equipment checklist

Cover the right side and answer each. If any stumps you, reread its section before the parent note.

What does the bold letter $x$ stand for, in one