Visual walkthrough — Open problems and future directions

This page takes ONE result from Open problems and future directions — the claim that machine learning learns correlation, not causation — and builds it from absolute zero, in pictures.

The central formula the parent note states without deriving:

$$P(Y\mid X)=\sum _{Z}P(Y\mid X,Z)P(Z\mid X)\text{versus}P(Y\mid do(X))=\sum _{Z}P(Y\mid X,Z)P(Z)$$

By the end you will see exactly why the two lines differ by a single factor, and why that single factor is the entire reason causal inference is an open problem.

We assume you know nothing except: a probability is a number between 0 and 1 that tells you how often something happens.

---

Step 1 — What a probability picture even looks like

WHAT. Before any formulas, we draw the world as a box of dots. Each dot is one day. We colour and place dots by three facts about that day.

WHY. Every symbol below is going to be a count of dots. If you can see the dots, you never have to trust a formula on faith — you can literally count.

PICTURE. Look at the figure. The three facts we track are:

• ==$X$== = "did people buy ice cream that day?" (dot on the right = yes)
• ==$Y$== = "did someone drown that day?" (dot is red = yes)
• ==$Z$== = "was it hot that day?" (dot in the top band = yes)

---

Step 2 — Building $P(Y\mid X)$ by counting dots

WHAT. We compute the plain observational probability: given that ice cream was bought, how often was there a drowning?

WHY. This is the ONLY thing a standard ML model can learn. It sees days, it counts, it reports the fraction. The symbol $P(Y\mid X)$ is defined to be exactly this fraction.

PICTURE. Take only the right-hand dots (ice cream = yes). Among those, count the red ones.

$$\underset{\text{fraction of red}}{\underbrace{P(Y\mid X)}}=\frac{\stackrel{\text{red dots on the right}}{\overbrace{\#(Y\text{ and }X)}}}{\underset{\text{all dots on the right}}{\underbrace{\#(X)}}}$$

Notice: this number is high in our picture. Ice-cream days really are drownier days. An ML model would honestly report a strong link — and be dangerously misleading.

---

Step 3 — Splitting the count by the hidden cause $Z$

WHAT. We slice the same right-hand dots into two piles: hot ($Z=$ yes, top) and cool ($Z=$ no, bottom), then re-combine.

WHY. We want to expose where the high number came from. The rule that lets us split-and-recombine is the law of total probability: any group's fraction is the weighted average of its sub-groups' fractions.

PICTURE. The right-hand column becomes two boxes stacked vertically.

$$P(Y\mid X)=\sum _Z\underset{\begin{array}{c}\text{red fraction}\\ \text{inside one slice}\end{array}}{\underbrace{P(Y\mid X,Z)}}\cdot \underset{\begin{array}{c}\text{how big that slice is}\\ \text{among ice-cream days}\end{array}}{\underbrace{P(Z\mid X)}}$$

Term by term:

• $P(Y\mid X,Z)$ — pick one slice (say "hot and ice cream"), the red fraction inside it.
• $P(Z\mid X)$ — the weight: of all ice-cream days, what fraction were hot?
• ${\sum }_Z$ — add up the two slices (hot, cool).

---

Step 4 — What an intervention physically does

WHAT. Now we stop observing and start acting. We force everyone to buy ice cream — hot day or not — by decree. This is written $do(X)$.

WHY. A policy question ("ban ice cream to stop drowning") is not "what do ice-cream days look like?" It is "what happens if I reach in and set ice cream, breaking its natural causes?" The symbol $do(\cdot )$ names that reach-in.

PICTURE. We snip the arrow $Z\to X$. Heat no longer decides ice cream; we do. So among the people we forced, hot and cool days appear in their natural proportion, not the ice-cream-biased one.

The key visual difference from Step 3: the slices are still there, the red fractions inside each slice are unchanged (heat still causes drowning!), but the weights are now the natural $P(Z)$, not the biased $P(Z\mid X)$.

---

Step 5 — The one-symbol difference

WHAT. We write the intervention formula next to the observation formula and stare at the single term that changed.

WHY. This is the punchline of the whole page. Everything above was to make this comparison undeniable.

PICTURE. Same two stacked slices, same inside-fractions; only the weight-bars resize.

$$\begin{array}{lcl}P(Y\mid X)& =& \sum _{Z}P(Y\mid X,Z)P(Z\mid X)\\ P(Y\mid do(X))& =& \sum _{Z}P(Y\mid X,Z)P(Z)\end{array}$$

• The parts in black are identical: $P(Y\mid X,Z)$, the honest "inside a slice heat still causes drowning" fractions.
• The parts in red differ: $P(Z\mid X)$ (heat given you saw ice cream — biased upward) versus $P(Z)$ (heat overall — the truth).

---

Step 6 — The degenerate cases (so you never get surprised)

WHAT. We check the two extreme worlds where the confusion vanishes.

WHY. The contract: never leave a reader in a scenario you didn't draw. Two cases collapse the gap to zero.

Case A — no confounder ($Z$ affects $Y$ but not $X$). Then $Z$ is independent of $X$, so $P(Z\mid X)=P(Z)$. Red bars equal black-neutral bars; the gap in Step 5's formula is exactly $0$. Correlation equals causation. This is the (rare) world where ordinary ML is already safe.

Case B — $X$ perfectly determines $Z$ (or vice versa). Then $P(Z\mid X)$ is $1$ for one slice and $0$ for the other; a single slice survives, and the "average" is degenerate. The formula still holds — it just reduces to one term.

---

The one-picture summary

The whole walkthrough in a single frame: two identical slices, identical red-fractions inside; the only thing an intervention edits is the weights, and that reweighting is the entire difference between "ice-cream days look drowny" and "banning ice cream does nothing".

Recall Feynman retelling — say it back in plain words

We drew every day as a dot: right for ice cream, red for a drowning, top for hot. When we asked "of ice-cream days, how many were drownings?" the answer came out scarily high — but only because hot days sneak into the ice-cream pile in huge numbers, and hot days are red for a totally separate reason (people swim). So the redness leaked in through which days got counted, not through ice cream doing anything. Then we imagined forcing everyone to buy ice cream regardless of weather. Now the hot and cool days sit in their natural proportion, the sneaky over-counting is gone, and the drowning rate doesn't budge. The math said the same thing: both formulas share the honest inside-piece $P(Y\mid X,Z)$, and differ by exactly one weight — $P(Z\mid X)$ (biased) versus $P(Z)$ (true). Machines only ever see and count dots, so they only ever get the biased weight. To get the true one you must either reach in and act, or hand the machine a map of which causes feed which.

Central result ::: $P(Y\mid X)={\sum }_{Z}P(Y\mid X,Z)P(Z\mid X)$ but $P(Y\mid do(X))={\sum }_{Z}P(Y\mid X,Z)P(Z)$ The single thing an intervention changes ::: the confounder weight $P(Z\mid X)\to P(Z)$; it cuts arrows into $X$ When the two are equal ::: when $Z$ is not a confounder, i.e. $P(Z\mid X)=P(Z)$ Why infinite data can't fix it ::: infinite observation nails $P(Y\mid X)$ perfectly, which is still the wrong quantity; you need intervention or causal assumptions

See also Statistical Learning Theory for why more data reduces estimation error but never touches this identification gap, and Bayesian Methods for encoding the causal graph as prior structure.