4.9.15 · D5Probability Theory & Statistics

Question bank — Central Limit Theorem — statement, proof sketch, significance

1,843 words8 min readBack to topic

True or false — justify

The CLT makes the raw data normal as grows
False. The data keeps its original shape forever; only the sampling distribution of the mean or sum (a derived quantity) becomes normal.
The must be normally distributed for the CLT to apply
False. The entire power of the theorem is that the inputs can be anything — dice, coins, exponentials — as long as their variance is finite; normality is an output, not a requirement.
If each is already , then is exactly , not just approximately
True. A sum of independent normals is exactly normal, so no limit is needed — the CLT's approximation becomes an equality for every finite .
The CLT guarantees the sample mean converges to
False — that is the Law of Large Numbers. The CLT is a finer statement about the shape of the fluctuations around once you magnify them by .
Doubling your sample size halves your standard error
False. The standard error is , so to halve it you need to quadruple — error shrinks like , not .
The CLT applies to the Cauchy Distribution once is large enough
False. The Cauchy has infinite variance (and no defined mean), so a core hypothesis fails; the average of Cauchy variables is still Cauchy with the same spread, no matter how large is.
Independence of the can always be dropped as long as is huge
False. The classical CLT needs independence; strongly dependent variables can converge to a non-normal limit or fail to stabilize. Weak-dependence versions exist but require extra conditions.
The variance of grows like , while its standard deviation grows like
True. so — this growth is exactly why we divide by to stabilize the limit.
A single large sample's histogram becoming bell-shaped is evidence of the CLT
False. That histogram estimates the original distribution of . The CLT is about the histogram of many averages, each computed from its own fresh sample.
The limit in the CLT depends on the third and fourth moments of
False. The limiting shape uses only the first two moments; higher moments (via skewness) affect only the speed of convergence, as quantified by Berry–Esseen below.

Spot the error

" is skewed, so can't be treated as normal."
The skew lives in the individual ; averaging 50 of them cancels most of the skew, and is well-approximated by a bell. Skewness slows convergence but does not forbid it for finite variance.
"For with , I standardize the cutoff directly."
You forgot the continuity correction. A discrete count is being covered by a continuous curve, so use (the right edge of the bar) to capture its width; ignoring it biases the probability.
"The CLT says , and since , the limit is a spike at ."
You mixed two different limits. As a distribution does collapse to (that's LLN); the CLT rescales by first so the spread stays fixed at , giving the non-degenerate .
"To use the CLT I only need a large , so always works."
The "" rule of thumb assumes mild skew. Heavily skewed or heavy-tailed distributions may need hundreds or thousands of terms before the approximation is good.
"Because , the probability equals exactly for large ."
Convergence in distribution means — it approaches but never equals for any finite . The error is small, not zero.
"CFs multiply for sums, so ."
Independence makes characteristic functions multiply, giving a power , not a product with . The is an exponent, not a coefficient.
"In the Taylor step we drop the term because it's small."
We drop it because we standardized , making exactly. It vanishes by construction, not by being negligible.

Why questions

Why do we divide by specifically, not or ?
Dividing by over-squashes the deviation to (the LLN regime); dividing by lets it blow up to infinity. Only , the actual standard deviation of the sum, holds the spread fixed for a stable limit.
Why do we use characteristic functions instead of working with densities directly?
For independent sums, densities combine by messy convolution, but characteristic functions simply multiply. The CF is also a unique fingerprint, and pointwise CF convergence implies distributional convergence via Lévy's continuity theorem.
Why does the CLT proof only need the first two moments?
The Taylor expansion of near is (here is the expansion's dummy argument); the constant term and the term (fixed by mean , variance ) survive the -th power limit after , while everything higher is swept into .
Why does the limit become ?
It's the classic exponential limit with . This is why the bell curve, whose CF is , emerges no matter what the inputs were.
Why does a heavy-tailed distribution converge more slowly (or not at all)?
Rare extreme values dominate the sum and refuse to average out. If the tail is so heavy that , there is no stabilizing scale and the CLT genuinely fails, as with the Cauchy Distribution.
Why can we approximate a binomial by a normal at all?
A variable is a sum of i.i.d. Bernoulli trials — exactly the setup the CLT needs — so its distribution approaches a bell (the de Moivre–Laplace theorem).
Why do confidence intervals and z-tests rely on the CLT?
They assume the standardized sample mean is approximately , which is precisely what the CLT delivers even when the underlying data are non-normal.
Why does the normal appear so universally in measurement noise?
Measurement error is typically a sum of many tiny independent effects; the CLT says such sums are approximately Gaussian regardless of each effect's individual distribution.
How fast does approach normality — is there a bound?
Yes: the Berry–Esseen theorem says , where is the third absolute moment and . The error shrinks like and grows with skewness — the quantitative reason skewed data need larger .

Edge cases

What happens to the CLT if ?
Then every equals with probability one; there is no randomness to average, is undefined (division by zero), and the theorem's hypothesis is violated.
What if the have finite mean but infinite variance?
The classical CLT does not apply. The properly scaled sum may converge to a non-Gaussian stable law instead of a normal, and the scaling is generally wrong.
Does the CLT hold if the are independent but not identically distributed?
Yes, under an extra uniformity condition — the Lindeberg–Feller Condition — which ensures no single term dominates the sum. Without it, one large-variance term can prevent normality.
What is the limiting distribution of the average of Cauchy variables as ?
It stays exactly Cauchy with the same scale — it does not concentrate and never becomes normal, a dramatic failure of both the Law of Large Numbers and the CLT.
For very small (say ) of non-normal data, is approximately normal?
Generally no. With so few terms the shape of the original distribution still dominates; the bell only emerges as grows, faster for symmetric light-tailed inputs and slower for skewed ones.
If the data are already normal, how large must be for the approximation to be good?
Any — a sum or mean of independent normals is exactly normal, so no minimum sample size is needed in this special case.
Why does a discrete (lattice) variable need a continuity correction but a continuous one does not?
A lattice distribution puts all its mass on evenly spaced points (e.g. integers), so its CDF is a staircase. Fitting a smooth normal curve to a staircase means each jump should be split at the midpoint, i.e. shift the cutoff by . A non-lattice continuous variable already has a smooth CDF, so no such correction is needed.