Joint, marginal, and conditional distributions: Visual walkthrough

Step 1 — A table of squares is the whole story

WHAT. Picture a rectangle chopped into a grid. Each small square is one combination of two facts. Let us call the first fact $X$ ("is this email spam?") and the second fact $Y$ ("does it contain the word free?"). Each of these can only be $0$ (no) or $1$ (yes).

WHY. Before any formula, probability is just area. If we shade each square so that its area equals the fraction of all emails falling in that box, then the whole rectangle has area $1$ (it is 100% of the emails). This picture — total area $1$ , split into labelled boxes — is called the joint distribution, written $p_{X, Y} (x, y)$ .

PICTURE. In the figure, the four boxes carry the numbers from the parent note. Read $p_{X, Y} (x, y)$ as: "the area of the box in row $x$ , column $y$ ."

Figure — Joint, marginal, and conditional distributions

Term-by-term, the total-area law reads $over both rows x \sum over both columns y \sum area of one box p_{X, Y} (x, y) = 100% of emails 1 .$ The symbol $\sum$ ("sigma") just means "add up all the terms as the index runs over its values."

Step 2 — Push a whole row into one pile → the marginal

WHAT. Take the spam row ( $X = 1$ ) and slide its two boxes together into a single bar. Its length is $0.05 + 0.25 = 0.30$ .

WHY. By sliding the boxes together we have thrown away the column information (we no longer care whether the word "free" appeared). What survives is "how much area is spam, regardless of $Y$ ." That surviving number is the marginal $p_{X} (x)$ . It is called marginal because in a printed table it lands in the margin — the row total written off to the side.

PICTURE. The teal bar on the right is the collapsed row; look how its length equals the two orange boxes stacked end to end.

$area of the whole row p_{X} (x) = sweep across columns y \sum one box’s area p_{X, Y} (x, y) .$

Numerically here: $p_{X} (1) = 0.05 + 0.25 = 0.30$ , and $p_{X} (0) = 0.60 + 0.10 = 0.70$ . The two row-totals themselves add to $1$ , so the marginal is a perfectly good distribution on its own.

Step 3 — Pick one column and stare at it (the slice)

WHAT. Now do the opposite of pushing: we keep only the column where $Y = 1$ ("free" appeared). That column has two boxes: ham-and-free $= 0.10$ and spam-and-free $= 0.25$ .

WHY. We want to answer "given that the word free showed up, how likely is spam?" The phrase "given that free showed up" means: forget every email that did not contain free. Visually, we grey out every box outside that column and zoom into the strip that remains.

PICTURE. Only the $Y = 1$ column stays coloured; the rest of the grid is faded. The two live boxes have areas $0.10$ (ham) and $0.25$ (spam).

Here is the crucial observation. The surviving strip has total area $0.10 + 0.25 = 0.35,$ which is exactly the column marginal $p_{Y} (1)$ . It is not $1$ . So the slice, as it stands, is not yet a valid distribution — its pieces do not add up to a full 100%.

Step 4 — Rescale the slice to 100% → the conditional

WHAT. Stretch the surviving strip so that its total area becomes $1$ again. To turn $0.35$ into $1$ we divide every box in the strip by $0.35$ .

WHY. "Given free appeared" defines a new, smaller world: the only emails that exist now are the $35%$ that contained free. Inside that world, probabilities must again sum to $1$ (it is the whole of this world). Dividing every piece by the strip's total $p_{Y} (1)$ is exactly the rescaling that restores the total to $1$ . The result is the conditional $p_{X ∣ Y} (x ∣ y)$ .

PICTURE. The short strip of Step 3 is stretched upward to full height $1$ ; the two pieces keep their ratio but now their lengths are $0.286$ and $0.714$ .

$share of the strip p_{X ∣ Y} (x ∣ y) = \frac{p _{X, Y} ( x , y ) box in the strip}{whole strip’s area p _{Y} ( y )} .$

The tall bar $∣$ inside $p_{X ∣ Y} (x ∣ y)$ is read "given". Everything to the right of the bar is the world we have zoomed into; everything to the left is the question we ask inside it.

Plugging in the spam numbers: $p_{X ∣ Y} (1 ∣ 1) = \frac{p _{X, Y} ( 1 , 1 )}{p _{Y} ( 1 )} = \frac{0.25}{0.35} \approx 0.714.$

Step 5 — Read it the other way → the product (chain) rule

WHAT. Take the conditional equation and multiply both sides by the marginal to undo the division.

WHY. The picture says a box's area = (its share of the strip) $\times$ (the strip's whole area). That is a rectangle's area = width-fraction $\times$ total — pure geometry. Written in symbols it is the product rule, and it lets us rebuild the joint from a marginal and a conditional (the direction generative models and Probabilistic graphical models actually use).

PICTURE. The stretched strip (height $1$ ) shrunk back down by the factor $p_{Y} (1) = 0.35$ returns to the original box areas — multiplication is the inverse of the rescaling.

$box area p_{X, Y} (x, y) = share inside the strip p_{X ∣ Y} (x ∣ y) \times strip total p_{Y} (y) .$

Because a box can be reached either by row-first or column-first, there are two factorings: $p_{X, Y} (x, y) = p_{Y ∣ X} (y ∣ x) p_{X} (x) = p_{X ∣ Y} (x ∣ y) p_{Y} (y) .$ Setting the two right-hand sides equal and dividing by $p_{Y} (y)$ gives Bayes' theorem: $p_{X ∣ Y} (x ∣ y) = \frac{p _{Y ∣ X} ( y ∣ x ) p _{X} ( x )}{p _{Y} ( y )} .$

Step 6 — The direction trap ( $p (X ∣ Y)$ ≠ $p (Y ∣ X)$ )

WHAT. Slice the other way — by row instead of column — and rescale that.

WHY. Zooming into the spam row ( $X = 1$ , total $0.30$ ) answers a different question: "given spam, how often is free present?" Its denominator is the row total $0.30$ , not the column total $0.35$ . Same box on top ( $0.25$ ), different strip underneath ⇒ different answer.

PICTURE. Left panel: column strip ÷ $0.35$ gives $0.714$ . Right panel: row strip ÷ $0.30$ gives $0.833$ . Same numerator box $0.25$ , two different denominators.

$p_{Y ∣ X} (1 ∣ 1) = \frac{0.25}{0.30} \approx 0.833 \neq = 0.714 = p_{X ∣ Y} (1 ∣ 1) .$

Step 7 — Degenerate case: an empty strip

WHAT. Suppose some value $y^{⋆}$ never occurs, so its whole column is blank: $p_{Y} (y^{⋆}) = 0$ .

WHY. The conditional would ask us to divide by $0$ . Geometrically there is no strip to rescale — you cannot stretch an area of $0$ up to $1$ . The event "given $y^{⋆}$ " describes a world that has no inhabitants, so "how likely is spam in it?" is meaningless.

PICTURE. A ghost column of zero width; the stretch arrow has nothing to grab, marked "undefined".

That is exactly why every definition carries the guard $p_{Y} (y) > 0$ . If a value can never happen, we simply do not condition on it.

Step 8 — Special case: the boxes were a perfect product (independence)

WHAT. Now take the continuous joint from the parent note, $f_{X, Y} (x, y) = 4 x y$ on the unit square, and slice it at a fixed $x$ .

WHY. When we compute the conditional $f_{Y ∣ X} (y ∣ x) = \frac{4 x y}{2 x} = 2 y$ , the $x$ cancels. Whatever value of $X$ we condition on, the shape over $Y$ is the same curve $2 y$ . Visually the vertical slices are all the same profile, just scaled — knowing $X$ tells you nothing new about $Y$ . That is exactly independence: the joint factors as $f_{X} (x) f_{Y} (y) = 2 x \cdot 2 y$ . See Independence and conditional independence.

PICTURE. Three vertical slices of the surface $4 x y$ at $x = 0.3, 0.6, 0.9$ ; after each is rescaled to area $1$ , all three collapse onto the identical line $2 y$ .

$f_{Y ∣ X} (y ∣ x) = \frac{4 x y joint at ( x , y )}{f _{X} ( x ) 2 x} = 2 y (no x left) ⟹ X ⊥ Y .$

Contrast this with the spam table, where the word free pushed spam from $0.30$ up to $0.714$ — there the slices had different shapes, so $X$ and $Y$ were dependent. (This shape-comparison is the honest test; a mere zero-correlation is not enough.)

The one-picture summary

Everything above is a single loop: push a row to get the marginal, cut a column to get a slice, divide by that column's total to rescale the slice into a conditional, and multiply back to recover the joint.

Recall Feynman retelling — the whole walkthrough in plain words

Start with a rectangle cut into little boxes, one box per (spam?, free?) combination, shaded so all the boxes together fill exactly 100% of the emails — that is the joint. Slide all the boxes in the spam row into one pile: you get "how much is spam, ignoring words" — the marginal. Now zoom into just the free column and grey out everything else; those boxes only add up to $0.35$ , not to a full 100%, so they're not a real world yet. Stretch that strip until it fills 100% again — i.e. divide every box by $0.35$ — and now each box tells you its share inside the free-only world: that is the conditional, and spam turns out to be $0.714$ . Read the stretch backwards (multiply instead of divide) and you rebuild any box's area from a share times a strip-total — the product rule — and because you can slice by row or column, equating the two ways of rebuilding gives Bayes. Watch the denominator: slice the column and you get $0.714$ ; slice the row and the same $0.25$ box over a different total $0.30$ gives $0.833$ — never swap those. If a column is empty you can't stretch nothing to 100%, so we forbid dividing by zero. And if every column-slice has the same shape after stretching, the two facts don't inform each other — that's independence.

Active recall

Why does the column-slice not already form a valid distribution?

Its boxes sum to the column marginal

p_{Y} (y)

, not to

1

What single operation turns the slice into the conditional, and why that one?

Divide every box by

p_{Y} (y)

; it is the only factor that rescales the strip's total back to

1

Same

0.25

box: why is

p (spam ∣ free) = 0.714

but

p (free ∣ spam) = 0.833

Different denominators — column total

0.35

vs row total

0.30

f_{Y ∣ X} (y ∣ x) = 2 y

, what does the missing

x

signify?

The conditional shape is independent of

x

, so

X

and

Y

are independent.

When is conditioning undefined, geometrically?

When the strip has area

0

(

p_{Y} (y) = 0

) — you cannot stretch nothing to total

1

Connections

Hinglish version of the parent
Bayes' theorem — the two factorings of the joint set equal.
Independence and conditional independence — identical slice shapes (Step 8).
Marginalization and the law of total probability — the "push a row" move (Step 2).
Covariance and correlation — a weaker summary than the full slice test.
Naive Bayes classifier — builds the joint by multiplying conditionals back (Step 5).
Probabilistic graphical models — choose which product-rule factoring to store.
Expectation and variance — averages taken against these very distributions.