Independence of random variables — formal definition: Foundations

This page assumes you have seen none of the notation in the parent note. We build every symbol from the ground up, in an order where each one leans only on the ones before it.

0. The starting picture: outcomes and probability

Imagine a fair experiment — a spinning wheel, a rolled die, a measured height. Every time you run it, one thing happens. That "one thing that happens" is called an outcome.

Why the topic needs it: independence is a statement about probabilities, so we need the object $P$ that hands out those numbers before anything else.

1. Events and the symbol $\cap$

The symbol $\cap$ means "and" — the overlap of two events.

Figure — Independence of random variables — formal definition

(Figure s01) — Two event-circles $A$ (lavender) and $B$ (coral). The picture makes the symbol $\cap$ concrete: the shaded lens where the circles overlap is $A \cap B$ , the region $P$ measures when we ask "both happen".

Why the topic needs it: this multiply-rule for events is the seed. The whole idea of independent variables is: make this rule hold for every question you could ask. See Independence of events.

2. Random variable $X$ — from outcomes to numbers

(Figure s02) — The three-box flow outcome → X → number shows literally what the symbol $X$ does: a die-outcome enters the butter-coloured machine $X$ and a bare number leaves. This is why ${X \in A}$ below is an event — it names all outcomes whose exiting number lands in $A$ .

The notation ${X \in A}$ (with $A$ now a set of numbers, like the interval $(- \infty, x]$ ) is itself an event: "the number $X$ produces lands inside $A$ ."

Why the topic needs it: independence of variables is defined by demanding ${X \in A}$ and ${Y \in B}$ be independent events for all number-sets $A, B$ .

3. Distribution, CDF, PMF, PDF — the languages of one variable

A random variable's full personality is its distribution: how its output-numbers spread out. There are three languages for it.

(Figure s03) — The same variable in three languages side by side: the lavender CDF climbing $0 \to 1$ , the coral PMF as discrete spikes, and the mint PDF where a shaded slice of area is a probability. Reading all three off one picture is what lets you recognise the parent's CDF-, PMF- and PDF-factorisation rules as one idea.

Why the topic needs it: the parent's three criteria (CDF, PMF, PDF factorisation) are the same statement written in these three languages. You must recognise all three.

4. Two variables at once: joint and marginal

Now the star of the show. With two machines $X$ and $Y$ running on the same experiment, one outcome produces a pair $(X, Y)$ — a point in the plane.

(Figure s04) — The lavender cloud is the joint pair $(X, Y)$ . Squashing every point straight down onto the $x$ -axis (coral) forgets $Y$ and leaves the marginal of $X$ ; squashing left onto the $y$ -axis (mint) leaves the marginal of $Y$ . The two shadows are exactly what the marginal integrals/sums below compute.

5. The symbols $⊥$ , product, and "support"

(Figure s05) — Left, a mint rectangle support: every horizontal slice offers the same range of $x$ , so fixing $y$ tells you nothing new about $x$ . Right, a coral triangle support: picking a large $x$ forbids small $y$ . This picture is why "support must be a box" is a genuine extra condition, not decoration.

Why the topic needs it: the parent's "consequences" section shows independence forces $Cov = 0$ , and warns that the reverse fails. You need these before reading it. See Covariance and correlation.

6. How it all feeds the topic

Equipment checklist

What is an outcome, and what is the sample space

Ω

An outcome is one single result of the experiment;

Ω

is the set of all possible outcomes.

What does

A \cap B

mean, in a picture?

The overlap ("both happen") — the lens where two event-circles cross.

State the event-independence rule.

P (A \cap B) = P (A) P (B)

: chance of both is the product of chances.

What is a random variable?

A machine that reads an outcome and returns a number.

Why is

{X \in A}

an event?

It is the yes/no question "did

X

's number land in the set

A

?", so all event-rules apply.

Define the CDF

F_{X}

F_{X} (x) = P (X \leq x)

, the chance the output is at most

x

; it climbs from 0 to 1.

PMF vs PDF — which uses spikes, which uses area?

PMF

p_{X} (x) = P (X = x)

is spike-heights (discrete); PDF

f_{X}

is density where probability is area under the curve (continuous).

How do you get a marginal from a joint, discretely and continuously?

Discrete:

p_{X} (x) = \sum_{y} p_{X, Y} (x, y)

; continuous:

f_{X} (x) = \int f_{X, Y} (x, y) d y

(collapse the other variable).

State the independence criterion in CDF, PMF and PDF languages.

F_{X, Y} = F_{X} F_{Y}

;

p_{X, Y} = p_{X} p_{Y}

;

f_{X, Y} = f_{X} f_{Y}

— joint equals product of marginals.

What does

X ⊥ Y

mean and what surface-picture goes with it?

X, Y

independent; the joint object equals the product of the two marginal shadows.

What is the support, for discrete and for continuous variables?

The set of

(x, y)

where the pair can land: where

p_{X, Y} > 0

(discrete spikes) or

f_{X, Y} > 0

(continuous density).

Why must the support be a rectangle for the factorisation test?

Otherwise allowed values of one variable depend on the other (info leaks), breaking independence even if the formula splits.

Give the covariance formula and what it detects.

Cov (X, Y) = E [X Y] - E [X] E [Y]

; it detects linear co-movement (0 = no linear tendency).

Connections

Independence of Random Variables — Formal Definition
Independence of events
Joint distribution and marginals
Covariance and correlation
Conditional distributions and conditional independence