Independence of random variables — formal definition: Visual walkthrough

Before anything: two words we will use constantly.

Everything below lives on a flat picture: the horizontal axis is every value $X$ could take, the vertical axis is every value $Y$ could take. One dot $(x, y)$ = " $X$ came out $x$ and $Y$ came out $y$ ".

Step 1 — Two wheels that ignore each other

WHAT. Picture two spinning wheels. Wheel- $X$ can stop on any value along the bottom axis; wheel- $Y$ on any value up the side.

WHY. We need the mental model of "not talking to each other" before we can write it as maths. Independence is precisely: watching wheel- $X$ stop tells you nothing new about where wheel- $Y$ will stop.

PICTURE. In the figure, the shaded strip is the event " $X$ fell in $A$ " (a vertical band, because it constrains only the horizontal axis). The other strip is " $Y$ fell in $B$ " (a horizontal band). The rectangle where they overlap is " $X \in A$ and $Y \in B$ ".

Figure — Independence of random variables — formal definition

The whole derivation is one question: how big is that overlap rectangle's probability?

Step 2 — The independence equation, term by term

WHAT. We claim: for independent variables, the probability of the overlap rectangle is the width-band probability times the height-band probability.

$overlap rectangle P (X \in A, Y \in B) = vertical band P (X \in A) \times horizontal band P (Y \in B)$

Reading each symbol on the picture from Step 1:

$P (X \in A, Y \in B)$ — the shaded overlap (both conditions hold at once).
$P (X \in A)$ — probability of the vertical band alone, ignoring $Y$ .
$P (Y \in B)$ — probability of the horizontal band alone, ignoring $X$ .

WHY multiply? This is inherited straight from Independence of events: two events are independent exactly when their combined chance is the product. A random variable is just a factory of events " $X \in A$ ", so we demand the product rule for every band $A$ and every band $B$ .

PICTURE. The next figure shows why "for all $A, B$ " is not overkill: one lucky rectangle obeying the product rule does not force the others. Independence is a promise about every rectangle at once.

Step 3 — Shrink the bands to "everything up to $x$ "

WHAT. Testing all sets $A, B$ is infinite work. So pick the simplest possible bands: $A = (- \infty, x]$ (" $X$ is at most $x$ ") and $B = (- \infty, y]$ (" $Y$ is at most $y$ ").

WHY these bands. They are nested — as you slide $x$ right, the band only grows. That single family of "staircase" bands is enough to rebuild every region later (Step 6 explains why). We trade an infinite test for a checkable one.

Plug them into the Step 2 equation:

$lower-left corner box P (X \leq x, Y \leq y) = left of x P (X \leq x) \times below y P (Y \leq y)$

PICTURE. Now the overlap is the whole lower-left quarter anchored at the corner point $(x, y)$ : everything to the left of $x$ AND below $y$ .

Step 4 — Name the corner box: the CDF

WHAT. The probability of that lower-left box has a name.

Rewriting Step 3 with these names gives the flagship result:

WHY it's beautiful. The messy 2-D box collapses into two 1-D piles you can compute separately, then multiply.

PICTURE. The figure paints the corner box as the product of a left-strip fraction and a bottom-strip fraction — area = width $\times$ height, made of probability.

Step 5 — From piled-up boxes to densities

WHAT. A CDF piles probability up. To see the probability at a point, we ask how fast the pile grows — that's a derivative.

WHY a derivative, and why two of them. $\frac{\partial}{\partial x} F$ measures "how much probability sits in a thin vertical sliver at $x$ ". Doing it again in $y$ , $\frac{\partial ^{2}}{\partial x \partial y} F$ , isolates the probability in a tiny tile at $(x, y)$ . That tile-density is the joint PDF.

Differentiate both sides of the CDF factorisation. A product of a function of $x$ and a function of $y$ splits cleanly (the $x$ -derivative ignores the $y$ -factor and vice-versa):

$\frac{\partial ^{2}}{\partial x \partial y} [F_{X} (x) F_{Y} (y)] = f_{X} (x) F_{X}^{'} (x) f_{Y} (y) F_{Y}^{'} (y)$

PICTURE. Left: a smooth surface whose tile at $(x, y)$ equals (slice through $x$ ) $\times$ (slice through $y$ ). Right: a grid of spikes whose height is (row weight) $\times$ (column weight).

Step 6 — Why testing only corner boxes is ENOUGH

WHAT. We only checked the nested $(- \infty, x]$ bands. Does the product rule really hold for every weird region $A, B$ ?

WHY yes. Any band is built from corner boxes by adding and subtracting. " $a < X \leq b$ " $=$ (box up to $b$ ) $-$ (box up to $a$ ). Every Borel set is reached by stacking such pieces. Because the product rule survives these additions and subtractions, holding on the corner boxes forces it everywhere.

PICTURE. A rectangle strip is drawn as (big corner box) $-$ (left box) $-$ (bottom box) $+$ (corner double-counted back in) — the classic inclusion–exclusion of four boxes. If each box factorises, so does the strip.

Recall Why sufficient, not merely necessary

Corner boxes generate all Borel sets ::: so the product rule on them propagates to every region, making the CDF test equivalent to full independence.

Step 7 — The trap: separable formula, coupled support

WHAT. A formula can look like $g (x) h (y)$ yet the variables still gossip — if the shape of the support links them.

WHY it fails. Take $f_{X, Y} = 2$ on the triangle $0 \leq x \leq y \leq 1$ . The number $2$ is as separable as it gets ( $2 = 1 \cdot 2$ ). But the region is a triangle, not a rectangle: learning $X = 0.9$ forces $Y \geq 0.9$ . That is information transfer — the definition in Step 2 breaks for the band $B = (- \infty, 0.5]$ once $X > 0.5$ .

Check by computing marginals and multiplying: $f_{X} (x) = \int_{x}^{1} 2 d y = 2 (1 - x), f_{Y} (y) = \int_{0}^{y} 2 d x = 2 y,$ $f_{X} (x) f_{Y} (y) = 4 y (1 - x) \neq = 2 = f_{X, Y} .$

PICTURE. The triangle support (magenta) with a dashed vertical line at $x = 0.9$ : the allowed $y$ -values start at $0.9$ , not at $0$ . The rectangle test fails because no product-shaped box contains exactly this triangle.

Step 8 — The clean case (so you see success too)

WHAT. $f_{X, Y} (x, y) = e^{- x - y}$ on $x, y > 0$ .

WHY it works. $e^{- x - y} = e^{- x} \cdot e^{- y}$ : separable, and the support $(0, \infty) \times (0, \infty)$ is a genuine rectangle (an infinite quarter-plane, still a product). Both boxes ticked.

$f_{X, Y} = f_{X} (x) e^{- x} \cdot f_{Y} (y) e^{- y} \Rightarrow X, Y \sim Exp (1) independently.$

PICTURE. The quarter-plane support (a real rectangle to infinity) with contour tiles that visibly factor into an $x$ -profile times a $y$ -profile.

Independence here means $E [X Y] = E [X] E [Y]$ , hence covariance $0$ — but remember from Covariance and correlation that zero covariance alone would not buy you independence back.

The one-picture summary

Everything on this page is one arrow of logic: event product rule → all bands → nested corner boxes → CDF → derivative → PDF/PMF, guarded by the rectangular-support gate.

Recall Feynman retelling (plain words)

Two fair-ground wheels are independent if watching one land tells you nothing about the other. On a flat map — wheel- $X$ across the bottom, wheel- $Y$ up the side — "both land in these ranges" is an overlap rectangle. Independence says: chance of the overlap = chance of the left band × chance of the bottom band, and this must hold for every rectangle, not one lucky one. Checking infinitely many rectangles sounds impossible, so we only check the simplest ones: "everything down-and-left of a corner point". That pile is the CDF, and the rule becomes $F_{X, Y} = F_{X} F_{Y}$ . Zoom into a single point with a double derivative and the corner boxes turn into tiny tiles: $f_{X, Y} = f_{X} f_{Y}$ . For grids of dots it's $p_{X, Y} = p_{X} p_{Y}$ . Two traps: a formula can look like (something in $x$ )(something in $y$ ) yet still be dependent if its support is a triangle rather than a box — then knowing $X$ pins down where $Y$ can live. And zero covariance is weaker than independence. The whole slogan: joint = product, everywhere, on a rectangle.

Connections

Independence of Random Variables — Formal Definition
Joint distribution and marginals
Independence of events
Conditional distributions and conditional independence
Covariance and correlation
Jointly Gaussian random variables
Sums of independent random variables — convolution