Joint distributions — joint PMF - PDF, marginal, conditional: Visual walkthrough

Step 1 — Two numbers at once: what a "point" means

WHAT. We have two random quantities. Call the first $X$ and the second $Y$ . A single outcome is not one number but a pair $(x, y)$ — a dot on a flat board. The horizontal position is the value of $X$ ; the vertical position is the value of $Y$ .

WHY. Before we can talk about "how $X$ and $Y$ move together" we need a stage where both live at once. One number lives on a line; two numbers live on a plane. That plane is the whole game.

PICTURE. Look at figure s01. The pale-yellow dot sits at $x = 3$ (three steps right) and $y = 2$ (two steps up). Reading the dot means reading both coordinates.

Step 2 — Piling up probability: the joint

WHAT. Now imagine running the experiment millions of times and dropping a marble at each observed $(x, y)$ . Some spots get tall stacks, some stay empty. The height of the stack over $(x, y)$ is how likely that pair is. That height function is the joint.

WHY. We want one object that stores everything. Not "how likely is $X = 3$ " and separately "how likely is $Y = 2$ ", but "how likely is the exact pair $(3, 2)$ ." The stack height answers that directly.

PICTURE. In figure s02 the board is a grid of buckets (discrete case). The number in each bucket is $p_{X, Y} (x, y)$ — its share of the marbles. Two rules jump out from the picture:

No bucket can hold a negative pile → $p_{X, Y} (x, y) \geq 0$ .
All buckets together hold the whole bag → the numbers sum to $1$ .

height over the bucket (x, y) p_{X, Y} (x, y) = share of marbles landing there P (X = x and Y = y), no negative piles p_{X, Y} \geq 0, whole bag x \sum y \sum p_{X, Y} = 1 .

Step 3 — Squashing the pile flat: the marginal

WHAT. Suppose I no longer care about $Y$ . I only ask: "how likely is $X = x$ , whatever $Y$ does?" To answer, I shove every marble in the row $X = x$ together into a single side-pile and read its total.

WHY. " $X = x$ , $Y$ anything" is the event ${X = x, Y = y_{1}}$ or ${X = x, Y = y_{2}}$ or … These sub-events never overlap (a marble has only one $y$ ), and together they are the whole row. Probability of a union of non-overlapping events = sum of their probabilities. That is the Law of Total Probability in action.

PICTURE. Figure s03 shows the arrows: each bucket in a row slides left into the margin — the strip written along the edge of the table. That edge total is the marginal. (That is literally where the word marginal comes from: it's written in the margin.)

margin total for row x p_{X} (x) = collapse all y y \sum each bucket in the row p_{X, Y} (x, y)

$\sum_{y}$ = "sweep across every $y$ and add." This is the collapse, not a deletion.
The result depends only on $x$ — $Y$ is gone.

Step 4 — Freezing a slice: why the raw slice is broken

WHAT. Now the opposite move. Someone tells me $Y = y_{0}$ exactly. I throw away every bucket except the column $Y = y_{0}$ and stare at what's left.

WHY. Knowing $Y = y_{0}$ shrinks the world: only outcomes with that $y$ are still possible. The column is the new set of possibilities.

PICTURE. Figure s04 grays out the whole board except one bright chalk-blue column. But look at the number written under the column — the leftover marbles add up to $p_{Y} (y_{0})$ , which is less than 1. A distribution must total 1. So this slice, as-is, is not yet a legal distribution.

x \sum buckets left in the column p_{X, Y} (x, y_{0}) = column total < 1 p_{Y} (y_{0}) \Rightarrow not yet a distribution.

Step 5 — Re-inflating the slice: the conditional

WHAT. Scale every surviving bucket by the same factor so the column totals $1$ again. The factor that does this is dividing by the column total $p_{Y} (y_{0})$ .

WHY. Start from the meaning of "given" — the Conditional Probability rule $P (A ∣ B) = \frac{P ( A \cap B )}{P ( B )}$ . Put $A = {X = x}$ and $B = {Y = y_{0}}$ : the intersection is the bucket $p_{X, Y} (x, y_{0})$ , and $P (B)$ is the column total. Dividing by the column total is exactly the re-inflation, and it keeps ratios between buckets intact.

PICTURE. Figure s05 stacks the tall re-scaled column beside the short raw one. Same shape, taller so it now fills a full unit of probability. The green check confirms it sums to $1$ .

re-inflated column, sums to 1 p_{X ∣ Y} (x ∣ y_{0}) = \frac{p _{X, Y} ( x , y _{0} ) one bucket in the column}{column total (the scale factor) p _{Y} ( y _{0} )}, p_{Y} (y_{0}) > 0.

Step 6 — Reading it backwards: the chain rule

WHAT. Multiply Step 5 across by the column total. The division undoes into a product.

WHY. Sometimes you build a joint from a story: "first draw $Y$ , then draw $X$ given $Y$ ." That story is a multiplication, and it's just Step 5 rearranged. Both orders must give the same joint.

PICTURE. Figure s06 shows the two-arrow factory: pick a column with weight $p_{Y} (y)$ , then within it pick a bucket with the re-inflated height $p_{X ∣ Y} (x ∣ y)$ ; their product rebuilds the original bucket.

original bucket f_{X, Y} (x, y) = shape inside the column f_{X ∣ Y} (x ∣ y) \cdot how heavy the column is f_{Y} (y) = f_{Y ∣ X} (y ∣ x) f_{X} (x) .

Step 7 — The degenerate case: independence

WHAT. Special situation: knowing $Y$ tells you nothing about $X$ . Then the re-inflated column looks identical for every $y$ — freezing $Y$ never changed the shape.

WHY. "Learning $Y$ changes nothing" means $f_{X ∣ Y} (x ∣ y) = f_{X} (x)$ . Feed that into the Step 6 chain rule: $f_{X, Y} = f_{X} \cdot f_{Y}$ . The joint factorizes into its two marginals — see Independence of Random Variables.

PICTURE. Figure s07 contrasts two boards. Left: every column has the same profile → the surface is a clean outer product, independent. Right: the profile tilts as $y$ changes → dependent. You literally see dependence as a column-to-column change of shape.

Independent ⟺ joint splits into a product f_{X, Y} (x, y) = f_{X} (x) f_{Y} (y) \forall x, y ⟺ column shape never changes f_{X ∣ Y} (x ∣ y) = f_{X} (x) .

Step 8 — Watch it run: the continuous triangle example

WHAT. Do the whole pipeline once on real dust: $f_{X, Y} (x, y) = 3 x$ on the triangle $0 < y < x < 1$ .

WHY. Every rule above, in order, on a case where the support matters — the sloped edge $y < x$ decides the integration limits.

PICTURE. Figure s08 draws the triangle. The horizontal blue arrow (for the marginal of $X$ ) sweeps $y$ from $0$ up to the sloped edge $y = x$ . The vertical pink arrow (for the marginal of $Y$ ) sweeps $x$ from the edge $x = y$ out to $1$ . Read the limits off the picture.

Marginal of $X$ (collapse $y$ , from $0$ to $x$ ): $f_{X} (x) = \int_{0}^{x} 3 x d y = 3 x \cdot x = 3 x^{2}, 0 < x < 1.$
Marginal of $Y$ (collapse $x$ , from $y$ to $1$ ): $f_{Y} (y) = \int_{y}^{1} 3 x d x = \frac{3}{2} (1 - y^{2}), 0 < y < 1.$
Conditional (divide, support shifts to $y < x < 1$ ): $f_{X ∣ Y} (x ∣ y) = \frac{3 x}{\frac{3}{2} ( 1 - y ^{2} )} = \frac{2 x}{1 - y ^{2}} .$

The one-picture summary

Figure s09 puts the whole journey on one board: the joint surface in the centre, marginals flattened onto each axis (the sum/integral arrows), one frozen column being lifted and re-inflated into the conditional, and the product-arrows showing the chain rule. This single diagram is the parent note.

Recall Feynman retelling — the whole walkthrough in plain words

Picture a board of buckets, each holding some marbles for a pair $(x, y)$ . That whole pile is the joint — it knows everything. Push every bucket in a row together and scribble the total in the margin: that's the marginal for $X$ — you didn't ignore $Y$ , you added over all of it. Now someone says " $Y$ is exactly $y_{0}$ ." Sweep away every bucket except that one column. But the leftovers don't fill a full bag anymore — they only add up to the column's own weight. So you scale them all up by dividing by that weight; now they fill a proper bag again. That re-inflated column is the conditional. Run the last step backwards — weight of column times shape inside it — and you rebuild any bucket: that's the chain rule. And if every column has the exact same shape no matter which $y$ you froze, then knowing $Y$ told you nothing about $X$ : the pile splits into a plain product of its two margins — that's independence. Three moves, one pile: jar, margins, slice-and-rescale.

Connections

Joint distributions — joint PMF - PDF, marginal, conditional — the parent this page derives.
Conditional Probability — the $P (A \cap B) / P (B)$ rule powering Step 5.
Law of Total Probability — the union-of-disjoint-events sum in Step 3.
Independence of Random Variables — the factorization of Step 7.
Bayes' Theorem — reversing the chain rule of Step 6.
Covariance and Correlation — how dependent surfaces (Step 7 right) get quantified.
Expectation and Variance — averaging over these distributions next.
Bivariate Normal Distribution — the smooth joint where every conditional is again normal.