Visual walkthrough — Conditional expectation

Before line one we owe you every symbol. Let us earn them.

---

Step 1 — What is a random variable, as a picture?

WHAT. A random variable $X$ is a rule that attaches a number to each possible outcome of an experiment. Roll a die: the outcome is a face, the number $X$ could be "the dots showing". A second variable $Y$ attaches another number to the same outcome.

WHY. Everything below lives on pairs $(X,Y)$ — two measurements of one experiment. To talk about $E[X\mid Y]$ we first need a picture where both live together.

PICTURE. Look at the grid. Each little cell is one possible pair of values: a value of $X$ (which row) and a value of $Y$ (which column). The amber number inside a cell is $P(X=x,Y=y)$ — the chance that this exact pair happens. This single object, the joint distribution, is our whole world.

Symbol tour of that cell weight:

• $X=x$ ::: "the first measurement landed on the value $x$" — picks a row.
• $Y=y$ ::: "the second measurement landed on $y$" — picks a column.
• the comma ::: means and — both happen at once — so we point at one cell.

See Joint and marginal distributions for the full object; we only need the grid.

---

Step 2 — Plain expectation $E[X]$: one giant weighted average

WHAT. $E[X]$ is your single best guess for $X$ before you know anything. It is the average of all $X$-values, each weighted by how likely it is.

$E[X]={\sum }_{x}x\cdot P(X=x)$

Term by term:

• $x$ ::: a value $X$ can take (a row label).
• $P(X=x)$ ::: the total chance of that row — sum the amber weights all the way across that row.
• ${\sum }_x$ ::: add this up over every row.
• the product $x\cdot P(X=x)$ ::: "value times its weight" — a tall bar's contribution.

WHY. This is the number the Tower Rule will hand back to us at the end. We must know exactly what it is so we recognise it when it appears.

PICTURE. Collapse every column into its row-total: the amber cells in a row merge into one white bar $P(X=x)$ on the right margin. Multiply each bar by its row-value $x$, add them — that single blended value is $E[X]$, drawn as the dashed cyan line.

---

Step 3 — Freeze one column: $E[X\mid Y=y]$ is a number

WHAT. Now someone whispers the value of $Y$. Say $Y=y$. We are no longer looking at the whole grid — only at one column. Inside that column we re-average $X$.

$E[X\mid Y=y]={\sum }_{x}x\cdot P(X=x\mid Y=y),P(X=x\mid Y=y)=\frac{P(X=x,Y=y)}{P(Y=y)}$

Term by term:

• $P(Y=y)$ ::: total weight of the whole column — add its amber cells top to bottom.
• dividing by $P(Y=y)$ ::: rescales the column so its weights add to $1$ again (it is now a valid distribution on its own).
• $P(X=x\mid Y=y)$ ::: the cell's share within its column — its amber weight as a fraction of the column total.
• ${\sum }_{x}x\cdot (\dots )$ ::: average the row-values using those rescaled column weights.

WHY. Conditioning = zoom into one column and forget the rest. The rescale by $1/P(Y=y)$ is the whole trick: without it the column's weights would not sum to $1$ and it would not be a probability distribution. (This is exactly the Conditional probability definition.)

PICTURE. One column is lit; the rest go dim. The amber cells inside it stretch to fill the column (that is the $÷P(Y=y)$ rescale). The cyan dot marks the balance point — the column's own average, $E[X\mid Y=y]$.

---

Step 4 — Unfreeze $Y$: now $E[X\mid Y]$ is a random variable

WHAT. Do Step 3 for every column. Each column $y$ gives its own average $g(y)=E[X\mid Y=y]$. Collect them: the function $g$ takes a column label and returns that column's mean. Feed it the random column $Y$ and you get a random variable $E[X\mid Y]=g(Y)$.

WHY. Before we knew $Y$, we did not know which column we would land in — so we do not know which column-average we will report. A quantity whose value depends on a random outcome is a random variable. This is the single hardest idea; the picture nails it.

PICTURE. A cyan balance-dot sits in every column. Together they trace a step-profile across the grid — one height per column. That whole profile is $E[X\mid Y]$: pick a random column and you read off a random height.

---

Step 5 — Average the column-averages, weighted by column size

WHAT. $E[X\mid Y]$ is itself a random variable, so it has its own expectation. Take it:

$E[E[X\mid Y]]={\sum }_{y}E[X\mid Y=y]\cdot P(Y=y)$

Term by term:

• $E[X\mid Y=y]$ ::: the cyan dot height of column $y$ (from Step 3).
• $P(Y=y)$ ::: the width/weight of that column (its total amber mass).
• ${\sum }_y(\text{height})\times (\text{weight})$ ::: the standard "expectation of a function of $Y$" recipe — see Expectation and its linearity.

WHY. We are averaging Step 4's step-profile. Crucially each column-average is weighted by how likely that column is, $P(Y=y)$ — a fat column pulls the blend toward its own dot. A plain unweighted average of the dots would be wrong unless all columns were equally likely.

PICTURE. The cyan dots (heights) get squashed together, each pushed with a force equal to its column's amber mass. The result is one dashed line — the average of the averages.

---

Step 6 — The collapse: two-step average = one-step average

WHAT. Now expand Step 5 back to cells and watch it become $E[X]$.

= \sum_y\sum_x x\,P(X=x,Y=y)$$ The $P(Y=y)$ we *divided by* in Step 3 is exactly the $P(Y=y)$ we *multiply by* in Step 5 — they **cancel**, un-rescaling the column back to raw amber weights. $$=\sum_x x\underbrace{\sum_y P(X=x,Y=y)}_{P(X=x)}=\sum_x x\,P(X=x)=E[X].$$ Term by term of the collapse: - $\tfrac{P(X=x,Y=y)}{P(Y=y)}\cdot P(Y=y)=P(X=x,Y=y)$ ::: rescale then un-rescale — the column is raw amber again. - $\sum_y P(X=x,Y=y)=P(X=x)$ ::: sweep across all columns in a row — **marginalising**, the very move from Step 2. - $\sum_x x\,P(X=x)=E[X]$ ::: and that is Step 2 exactly. **WHY.** Column-averaging then column-weighting **re-assembles every original cell untouched**, then sums them row-wise. Splitting the grid into columns and recombining cannot create or destroy weight — so the grand average is unchanged. **PICTURE.** The dim columns re-light to full amber (cancellation), then all columns sweep sideways into the row-margin bars of Step 2. The dashed cyan line lands on the *same* height as $E[X]$. ![[deepdives/dd-maths-4.9.13-d2-s06.png]] > [!formula] The Tower Rule, now earned > $$\boxed{\,E\big[E[X\mid Y]\big]=E[X]\,}$$ > Average within each column, then average the columns weighted by their size — you recover the average over the whole grid. --- ## Step 7 — The degenerate cases (never left in the dark) **WHAT.** Check the boundaries where the picture threatens to break. **Empty column, $P(Y=y)=0$.** Step 3 divides by the column mass. If a column is truly empty, $E[X\mid Y=y]$ is **undefined** — but it never enters Step 5, because its weight $P(Y=y)=0$ zeroes out the term. The grid simply has no such column. **One single column ($Y$ constant).** Then there is nothing to average across: $E[X\mid Y]=E[X]$ already, and the tower is the trivial statement $E[X]=E[X]$. **$X$ and $Y$ independent.** Every column has the **same** step-height, because knowing the column tells you nothing about $X$: $E[X\mid Y]=E[X]$, a flat profile. See [[Conditional probability]] for what independence does to the cells (each cell factorises $P(X=x)P(Y=y)$). **PICTURE.** Left panel: a flat cyan profile — independence, all columns agree. Right panel: a single fat column — $Y$ constant. In both the two-step average trivially equals the one-step average. ![[deepdives/dd-maths-4.9.13-d2-s07.png]] > [!formula] Independence collapse (special case, visible) > If $X\perp Y$: $E[X\mid Y]=E[X]$ — the flat profile. The tower still holds, now saying $E[\,\text{constant}\,]=\text{constant}$. --- ## The one-picture summary Everything in one frame: **column-average, then weight-and-recombine, land back on $E[X]$.** ![[deepdives/dd-maths-4.9.13-d2-s08.png]] > [!recall]- Feynman retelling — the whole walkthrough in plain words > Picture a classroom seating chart as a grid. Every seat holds a tiny weight = how likely that exact (score $X$, class $Y$) pair is (Step 1). Your best guess for a random kid's score, ignoring class, is the average over the whole chart, $E[X]$ (Step 2). > Now someone tells you the class. You zoom into that one **column**, blow it up so its weights add to one, and read its average — one number (Step 3). Do this for every class and you get a row of dots, one per column: that row of dots is $E[X\mid Y]$, and since you don't know in advance which class the kid is in, it is itself random (Step 4). > Blend those class-averages, giving each class a push equal to its size (Step 5). The blow-up you did in Step 3 and the size-weight in Step 5 are exact opposites — they cancel, un-shrinking each column back to its raw seats, and sweeping all seats together gives the whole-chart average again (Step 6). So averaging within classes then across classes lands on the same number as averaging every kid at once. Empty classes never matter (zero weight), one class is trivial, and if class is irrelevant every column-average is identical (Step 7). That is the Tower Rule — split, average, recombine, nothing lost. --- ## Connections - [[Conditional expectation]] (parent) - [[Joint and marginal distributions]] — the grid and its margins - [[Conditional probability]] — the column rescale in Step 3 - [[Law of total probability]] — the sibling identity for probabilities - [[Expectation and its linearity]] — the weighted-average recipe in Steps 2 & 5 - [[Conditional variance and Eve's law]] — what happens to *spread* when you condition - [[Wald's identity]] — the tower applied to random sums - [[Martingales]] — where $E[X\mid\mathcal F]$ generalises this