Foundations — Central Limit Theorem — statement, proof sketch, significance

0. What is a "random variable"?

Before any Greek letter, we need the object everything else describes.

The picture: imagine a machine with a lever. Pull it and a number pops out. You don't know which number in advance, but you know the rules of how likely each number is.

• A die roll: $X\in \{1,2,3,4,5,6\}$, each equally likely.
• A coin flagged $1$ for heads, $0$ for tails: $X\in \{0,1\}$ — this is the Bernoulli building block.

Why the topic needs it: the CLT is a statement about many such machines, so we must name one first.

1. The probability operator $P$, and how likelihood is stored

Before we can average outputs we need to say how likely each output is.

There are two flavours of machine, and they store their likelihood differently:

The picture: a PMF is a row of spikes (bars) whose heights sum to $1$; a PDF is a smooth curve whose total shaded area is $1$.

Why the topic needs it: every later formula weights outputs by their likelihood, so we must fix what "likelihood" means — and note it works for both discrete and continuous machines.

2. The mean $\mu$ and the symbol $E[\cdot ]$

The picture: the mean is the balance point of the probability picture — the spot where the bars (or the area under the curve) would balance on a pencil tip.

Why the tool $E[\cdot ]$ and not just "average of my data"? The average of data is a number that changes every time you sample. $E[X]$ is the true fixed target — a property of the machine, not of any one sample. The CLT is all about how the noisy data-average dances around this fixed $\mu$.

3. Variance ${\sigma }^2$ and standard deviation $\sigma$

The mean tells you where the machine centres. It says nothing about how wild the outputs are.

The picture: $\sigma$ is the typical "arm span" of the picture — how far a single output usually strays from the balance point.

Why the topic needs it: the CLT requires ${\sigma }^2$ to be finite and positive. If the spread is infinite (like the Cauchy Distribution), the whole theorem collapses — the mean never settles.

4. "i.i.d." — independent and identically distributed

This four-letter tag is the load-bearing assumption of the classical CLT. First fix the counter $n$:

The picture: a row of identical, separately-wired machines, each pulled once — no wire connects any two.

Why the topic needs it: independence is exactly what lets the proof factor a product of expectations (see Characteristic Functions), and it makes variances add. Relax "identically distributed" a little and you need the Lindeberg–Feller Condition instead.

5. The sample mean ${\bar{X}}_n$ and why its spread is $\sigma /\sqrt{n}$

Now the crucial fact the parent note asserts, built from the two variance rules of Section 3:

The picture: as you use more machines, the histogram of the average pulls in tight around $\mu$ — its arm span shrinks like $1/\sqrt{n}$.

Why $\sqrt{n}$ matters so much: because the spread shrinks (that's the Law of Large Numbers) but only slowly, we can't just subtract $\mu$ — we'd get a shrinking-to-zero object. We must rescale by exactly this standard error to keep a stable, non-trivial shape. That is the Standard Error that appears everywhere in the topic.

6. Standardizing: the symbol $Z_n$

Reading it in words: how many standard errors is my sample average away from the truth?

Why the topic needs it: the limit the CLT proves is a fixed shape, so we must strip out $\mu$ and $\sigma$ (which differ per problem). $Z_n$ is that stripped, universal version.

7. The target shape: $N(0,1)$, $\Phi$, and "converges in distribution"

The picture: $\varphi$ is the hill; $\Phi$ is the running total of shaded area as you sweep left to right — it climbs from $0$ up to $1$.

Why the topic needs it: this arrow is the exact meaning of the CLT's conclusion. Everything downstream — Confidence Intervals, Hypothesis Testing — z and t tests — reads off values of $\Phi$.

The prerequisite map

Equipment checklist

Give yourself a tick only if you can answer out loud: