Exercises — Central Limit Theorem — statement, proof sketch, significance

$P(Z>z)=1-\Phi (z)$? Total area under the bell is 1. $\Phi (z)$ is the area to the left of $z$. Everything not to the left is to the right, so the right-tail area is $1-\Phi (z)$. ::: $P(Z>z)=1-P(Z\le z)=1-\Phi (z)$.

The CLT needs three things: independent, identically distributed, and finite variance (${\sigma }^2<\infty$).

• (a) ✅ Dice rolls are i.i.d. with ${\sigma }^2=\frac{35}{12}<\infty$. The mean of 200 is beautifully normal.
• (b) ❌ The Cauchy distribution has no finite mean and infinite variance. The recipe breaks — the average of Cauchy draws is itself Cauchy, never settling. This is the classic CLT counterexample.
• (c) ✅ Heights are i.i.d. with finite variance, so ${\bar{X}}_{40}$ is approximately normal (and $n=40>30$, comfortably past the rule of thumb).

• Sample mean ${\bar{X}}_{64}$: centre $=\mu =10$, standard error $=\frac{\sigma }{\sqrt{n}}=\frac{4}{\sqrt{64}}=\frac{4}{8}=0.5$.
• Sum $S$: centre $=n\mu =64\cdot 10=640$, spread $=\sigma \sqrt{n}=4\cdot 8=32$. Notice the average's spread ($0.5$) is eight times smaller than a single value's spread ($4$) — that is $1/\sqrt{64}=1/8$ at work.

Standard error $=\frac{\sigma }{\sqrt{n}}=\frac{10}{\sqrt{100}}=1$. Standardize the boundary $52$: $z=\frac{{\bar{X}}_n-\mu }{\sigma /\sqrt{n}}=\frac{52-50}{1}=\mathrm{2.00.}$ Now convert the "greater than" to a table lookup. The table gives $\Phi (z)=P(Z\le z)$ — area to the left. We want the area to the right of $z=2$, which is everything minus the left part (total area is 1): $P({\bar{X}}_{100}>52)=P(Z>2.00)=1-\Phi (2.00)=1-0.9772=\mathrm{0.0228.}$ About a $2.3\%$ chance the average of 100 draws exceeds 52. (No symmetry was assumed here — just "left area + right area = 1.")

• $E[S]=n\mu =100\cdot 3.5=350$; $\mathrm{SD}(S)=\sigma \sqrt{n}=1.7078\sqrt{100}=17.078$.
• $z=\frac{S-n\mu }{\sigma \sqrt{n}}=\frac{330-350}{17.078}=-1.171$.
• For a negative $z$ we use the bell's mirror symmetry about 0: the left-tail area at $-1.171$ equals the right-tail area at $+1.171$, i.e. $\Phi (-1.171)=1-\Phi (1.171)$. Hence $P(S<330)=\Phi (-1.171)=1-\Phi (1.171)\approx 1-0.8792=\mathrm{0.1208.}$ About a $12\%$ chance the 100-roll total is below 330. (Here symmetry is used, and stated explicitly, because the table only lists positive $z$.)

A central 95% region on a bell curve reaches out to $z=1.96$ standard errors on each side (that is why $z_{0.025}=1.96$: it leaves $2.5\%$ in each tail). So $c=1.96\times (\text{standard error})=1.96\times 1=\mathrm{1.96.}$ Thus ${\bar{X}}_{100}$ lands in $(48.04,51.96)$ about 95% of the time. This is the seed of Confidence Intervals.

• $\mu =np=200(0.3)=60$; $\sigma =\sqrt{np(1-p)}=\sqrt{200(0.3)(0.7)}=\sqrt{42}=6.4807$.
• Continuity correction — where it comes from. The count $X$ can only be a whole number, so its probability lives on isolated spikes at $0,1,2,\dots$. To draw those spikes as area (so a smooth curve can approximate them), we widen each integer $k$ into a bar spanning $[k-0.5,k+0.5]$ — a rectangle of width 1 whose area equals $P(X=k)$. The smooth normal curve is then matched to the tops of these bars. Because the event $\{X\le 50\}$ includes the entire bar at $k=50$, its area runs up to that bar's right edge, $50.5$ — not to $50$. Look at the figure: the butter-coloured bar at 50 is fully inside the shaded region, and the lavender dashed line sits at its right edge. $z=\frac{50.5-60}{6.4807}=-\mathrm{1.466.}$ $P(X\le 50)\approx \Phi (-1.466)=1-\Phi (1.466)\approx 1-0.9287=\mathrm{0.0713.}$

Since $\mathrm{Var}(\sum (X_i-\mu ))=n{\sigma }^2$ and dividing a random variable by $n$ divides its variance by $n^2$: $\mathrm{Var}(W_n)=\frac{n{\sigma }^2}{n^2}=\frac{{\sigma }^2}{n}\stackrel{n\to \infty }{\to }0.$ The spread collapses to zero — $W_n\to 0$, a single point. That is the Law of Large Numbers, not a bell curve. Dividing by $n$ "squashes too hard." To keep a non-degenerate spread you must divide by $\sqrt{n}$, because that exactly cancels the $\sqrt{n}$ growth of the sum's standard deviation. This is the Goldilocks scaling.

First, what "skewness" means. Skewness is a single number measuring lopsidedness — how much longer one tail is than the other. Its formula is the standardized third moment: $\gamma =E\left[{\left(\frac{X-\mu }{\sigma }\right)}^3\right].$ A symmetric distribution has $\gamma =0$ (left and right cancel in the cube); a right-skewed one has $\gamma >0$ (the long right tail makes the cube positive on average). A fair coin's Bernoulli$(0.5)$ has $\gamma =0$; an exponential has $\gamma =2$. Now the answer: the skewed one (exponential) needs a larger $n$. The CLT's limit throws away everything beyond the first two moments, but for finite $n$ the leftover skewness distorts the bell. The approximation error shrinks like $\frac{\gamma }{\sqrt{n}}$ (the Berry–Esseen flavour), so a bigger $\gamma$ demands a bigger $n$ to reach the same closeness. Symmetric input ($\gamma =0$) converges fast; the exponential's $\gamma =2$ makes it slow. Exactly when a sum of many small independent (not necessarily identical) pieces still goes normal is pinned down by the Lindeberg–Feller Condition.

The 95% margin of error is $z_{0.025}\cdot \frac{\sigma }{\sqrt{n}}=1.96\cdot \frac{12}{\sqrt{n}}$. Require this $\le 2$: $1.96\cdot \frac{12}{\sqrt{n}}\le 2\Rightarrow \sqrt{n}\ge \frac{1.96\cdot 12}{2}=11.76\Rightarrow n\ge 11.76^2=\mathrm{138.3.}$ Round up (you can't have too few): $n=139$. Notice: to halve the margin from 2 to 1 you would need $\sqrt{n}$ twice as big, i.e. $4\times$ the data — the "$1/\sqrt{n}$ diminishing returns."

Each mean is approximately normal: $\mathrm{Var}({\bar{X}}_A)=\frac{{\sigma }_A^2}{n_A}=\frac{36}{50}=0.72,\mathrm{Var}({\bar{X}}_B)=\frac{{\sigma }_B^2}{n_B}=\frac{81}{80}=\mathrm{1.0125.}$ Variances of independent quantities add even for a difference (subtracting doesn't help the wobble): $\mathrm{Var}(D)=0.72+1.0125=1.7325,\mathrm{SD}(D)=\sqrt{1.7325}=\mathrm{1.3162.}$ This standard deviation is the engine of a two-sample z-test.

Under equal true means, $D$ is centred at 0 with $\mathrm{SD}(D)=1.3162$. Standardize: $z=\frac{3-0}{1.3162}=\mathrm{2.279.}$ Two-sided 5% cutoff is $|z|>1.96$. Since $2.279>1.96$, the observed gap is surprising — we would reject "means equal" at the 5% level. (Two-sided $p$-value $\approx 2(1-\Phi (2.279))\approx 0.0226$.)

Check the mean and variance by integrating the tail.

• Mean: $E[X]={\int }_1^{\infty }x\cdot \frac{2}{x^3}dx={\int }_1^{\infty }2x^{-2}dx=[-2x^{-1}{]}_1^{\infty }=2$. Finite. ✅
• Variance needs $E[X^2]={\int }_1^{\infty }{x}^2\cdot \frac{2}{x^3}dx={\int }_1^{\infty }2x^{-1}dx=[2\ln x{]}_1^{\infty }=\infty$. Diverges. ❌ Because ${\sigma }^2=\infty$, the classical (finite-variance) CLT does not apply. The sample mean does converge to 2 (mean exists, so LLN holds), but its fluctuations are not Gaussian and don't shrink like $1/\sqrt{n}$ — they follow a heavy-tailed stable law. Moral: "has a mean" is not enough; you need a finite variance.

First, a word on the notation $o(\cdot )$ ("little-o"). Writing $g(s)=o(s^2)$ means $g$ shrinks strictly faster than $s^2$ as $s\to 0$, i.e. $g(s)/s^2\to 0$. It is a bookkeeping tag for "terms too small to matter in the limit" — here, everything past the second moment. With scaling $n^{-a}$, the Taylor expansion of the characteristic function near 0 gives ${\phi }_Y(t/n^a)=1-\frac{t^2}{2n^{2a}}+o\left(n^{-2a}\right),$ so ${\phi }_{Z_n}(t)={\left(1-\frac{t^2}{2n^{2a}}+\cdots \right)}^n.$ For the limit $(1+\frac{c_n}{n}{)}^n\to e^c$ to be finite and non-zero we need the bracket's correction to behave like $(\text{constant})/n$, i.e. $n^{2a}=n$, hence $2a=1$, so $a=\frac{1}{2}$.

• If $a>\frac{1}{2}$: exponent $\to 0$, $\phi \to 1$, limit is a point mass (over-squashed → LLN).
• If $a<\frac{1}{2}$: correction blows up, $\phi \to 0$, no proper limit (under-squashed → diverges). Only $a=\frac{1}{2}$ lands on $e^{-t^2/2}$, the standard-normal fingerprint. This is the CF-level reason for $\sqrt{n}$.

• Sum's centre: $n\mu =2500\cdot (-0.05)=-125$.
• Sum's spread: $\sigma \sqrt{n}=1.10\cdot \sqrt{2500}=1.10\cdot 50=55$.
• Standardize the boundary $0$ using $z=\frac{S-n\mu }{\sigma \sqrt{n}}$: $z=\frac{0-(-125)}{55}=\frac{125}{55}=\mathrm{2.2727.}$
• We want the right-tail area (being ahead means $S>0$), so total minus left: $P(S>0)=P(Z>2.2727)=1-\Phi (2.2727)\approx 1-0.9885=\mathrm{0.0115.}$ About a 1.2% chance of being ahead after 2500 plays. The tiny per-play edge, multiplied by many plays and shrunk-relative-to-the-mean by the $\sqrt{n}$ scaling, makes losing almost certain. That is exactly why casinos profit: the CLT concentrates the aggregate around the negative mean.

To standardize an average, divide the deviation by what? ::: the standard error $\sigma /\sqrt{n}$ — never by $\sigma$ alone.