Visual walkthrough — Central Limit Theorem — statement, proof sketch, significance

4.9.15 · D2 · Maths › Probability Theory & Statistics › Central Limit Theorem — statement, proof sketch, significanc

---

Step 0 — "Random variable" kya hota hai, pictures mein?

Pehle sab se pehle: ek random variable bas ek aisa number hai jo aapko abhi pata nahi, kisi experiment se produce hota hai — dice ka face, kisi ajnabi ki height, bus aane ka time. Hum isse $X$ likhte hain. Iski distribution "har value kitni likely hai" ki picture hai — ek bar chart (discrete) ya ek smooth hill (continuous).

Do numbers us picture ko summarise karte hain:

Upar teen shapes dekho — ek dice (flat bars), ek exponential (right-leaning slide), ek coin (do spikes). Yeh ek dusre se bilkul alag lagte hain. Aage ka pura miracle yeh hai ki inhe average karo toh ek hi curve milti hai. Teeno apne dimag mein rakho.

---

Step 1 — Hum actually average study karte hain, data nahi

Hum yeh claim NAHI karte ki data bell ban jaata hai. Hum $n$ independent copies $X_1,X_2,\dots ,X_n$ lete hain (independent = ek outcome doosre ke baare mein kuch nahi batata) aur unka sample mean banate hain:

${\bar{X}}_n=\underset{\text{share equally}}{\underbrace{\frac{1}{n}}}\underset{\text{add all }n\text{ draws}}{\underbrace{\sum _{i=1}^n{X}_i}}.$

• ${\sum }_{i=1}^n{X}_i$ matlab "saare $n$ draws jodo" — bada $\Sigma$ Greek capital S hai, Sum ke liye.
• $\frac{1}{n}$ se multiply karna us total ko average mein badalta hai.

Average KYUN? Ek draw predict karna impossible hai. Lekin average lene se jo draws zyada hain aur jo kam hain woh cancel karte hain. Neeche ki picture aisi ek average ko scattered dots se banati hai.

KAISA DIKHTA HAI: scattered blue dots (individual draws, wildly spread) ek single yellow tick (unka average) par collapse hote hain jo true mean $\mu$ ke bahut paas hota hai. Yeh experiment dobara karo toh yellow tick thoda alag jagah padta hai — us yellow tick ki woh jitter is poore page ki star hai.

---

Step 2 — Pura experiment baar baar repeat karo → ek nayi distribution appear hoti hai

Yeh woh mental move hai jo sabko confuse karti hai. Isse freeze karo:

KYUN: har average khud ek random number hai (har baar jitter karta hai). Ek random number ki apni distribution hoti hai — aur wahi woh object hai jo CLT describe karta hai.

KAISA DIKHTA HAI: Step 1 ke bohot saare yellow ticks, ek histogram mein stack kiye hue. Chahe source (green exponential, skewed) ek taraf jhuka hua ho, averages ka dhher already symmetric aur bell-shaped ki taraf jhuk raha hai.

---

Step 3 — Woh dhher kahan baithta hai, aur kitna wide hai?

Shape bell hai yeh prove karne se pehle, hum uska center aur width pin down karte hain — kyunki ek bell ko dono chahiye.

$E[{\bar{X}}_n]=\mu ,\mathrm{Var}({\bar{X}}_n)=\frac{{\sigma }^2}{n},\mathrm{SD}({\bar{X}}_n)=\frac{\sigma }{\sqrt{n}}.$

• $E[{\bar{X}}_n]=\mu$: averages ka average phir bhi true mean hai — dhher sach par centered hai.
• $\mathrm{Var}({\bar{X}}_n)={\sigma }^2/n$: $n$ se divide karna spread ko shrink karta hai. KYUN $/n$ aur $/\sqrt{n}$ nahi variance ke liye? Kyunki independent cheezein ki variances add hoti hain, isliye $\mathrm{Var}(\sum X_i)=n{\sigma }^2$, aur kisi variable ko $\frac{1}{n}$ se multiply karne par uski variance $\frac{1}{n^2}$ se multiply hoti hai: $\frac{1}{n^2}\cdot n{\sigma }^2=\frac{{\sigma }^2}{n}$.
• $\mathrm{SD}({\bar{X}}_n)=\sigma /\sqrt{n}$: variance ki square root lo. Is width ka apna naam hai, standard error.

KAISA DIKHTA HAI: $n=1,10,100$ ke teen dhhers. Sab $\mu$ par centered; har ek pichle se $\sqrt{10}\approx 3.16\times$ narrow hai. Yeh narrow-hona-lekin-na-hilna $\sigma /\sqrt{n}$ ka visual signature hai.

---

Step 4 — Rescale karo taki width 1 par freeze ho (standardize)

Problem: Step 4 ki picture ke har dhher ki width alag hai, toh hum shapes compare nahi kar sakte. Ise standardizing se fix karo — center subtract karo, width se divide karo:

$Z_n=\frac{\stackrel{\text{distance from truth}}{\overbrace{{\bar{X}}_n-\mu }}}{\underset{\text{one standard error}}{\underbrace{\sigma /\sqrt{n}}}}.$

• Numerator ${\bar{X}}_n-\mu$: dhher ko slide karta hai taki uska center $0$ par ho.
• Denominator $\sigma /\sqrt{n}$: rescale karta hai taki width exactly $1$ ho.

Toh $Z_n$ hamesha mean $0$ aur standard deviation $1$ rakhta hai, chahe $n$ kuch bhi ho ya original $X$ kuch bhi ho. Ab har dhher ek hi ruler par draw hota hai aur hum pooch sakte hain: kya shape settle down hoti hai?

KYUN: shapes compare karne ke liye size differences hataani padti hain — jaise photos ko ek hi width par resize karo pehle compare karne se.

KAISA DIKHTA HAI: Step 4 ke teen alag-width dhhers, sab stretch/shift hokar center 0, width 1 par ek dusre ke upar baith jaate hain. Ab woh almost coincide karte hain — aur jo curve unhe touch karti hai woh Step 5 ka target hai.

---

Step 5 — "Fingerprint" trick: characteristic functions

Shape ek specific curve banti hai yeh prove karne ke liye, messy histograms ko directly compare karna impossible hai. Hum ek aisi tool par switch karte hain jo "independent randoms ko add karna" ko plain multiplication mein badal deti hai.

KYUN yeh tool aur densities nahi? Do independent cheezein ka sum unki densities ka convolution hota hai — ek messy smearing integral. Lekin unke fingerprints simply multiply hote hain: ${\phi }_{X+Y}(t)={\phi }_X(t){\phi }_Y(t)\text{(jab independent ho).}$ Baar baar smearing ki jagah multiplication ka limit lena kaafi aasaan hai.

KAISA DIKHTA HAI: yellow dot $e^{itX}$ unit circle par ride karta hua; red vector bohot saare aise dots ka average hai — woh shrunken averaged vector hi ${\phi }_X(t)$ hai. Jab $t\to 0$ toh dots barely spin karte hain, isliye unka average vector length $\approx 1$ rakhta hai.

---

Step 6 — $t=0$ ke paas zoom in karo: sirf mean aur variance bachte hain

Hamare standardized building block $Y=\frac{X-\mu }{\sigma }$ (mean $0$, variance $1$) ke liye, $t=0$ ke paas fingerprint ko uski pehli curvature se approximate kiya jaata hai — ek Taylor expansion:

${\phi }_Y(s)=\underset{\text{starts at }1}{\underbrace{1}}+\underset{=0\text{ since }E[Y]=0}{\underbrace{isE[Y]}}-\underset{=-s^2/2\text{ since }E[Y^2]=1}{\underbrace{\frac{s^2}{2}E[Y^2]}}+o(s^2).$

• ${\phi }_Y(0)=1$: non-spinning dots ka average length 1 deta hai.
• Linear term zero ho jaata hai kyunki humne center kiya ($E[Y]=0$).
• Quadratic term $-\frac{s^2}{2}$ hi ek surviving information hai: woh variance hai, jo $1$ hai.
• $o(s^2)$ matlab "junk jo $s^2$ se faster shrink karta hai" — $0$ ke paas negligible.

KYUN Taylor yahan? Kyunki agli step mein har input $s=t/\sqrt{n}$ tiny hoga, isliye sirf $\phi$ ka $t=0$ ke ekdum paas ka behaviour matter karta hai. Taylor precisely woh microscope hai "ek point ke paas ke liye".

KAISA DIKHTA HAI: true fingerprint curve aur parabola $1-\frac{s^2}{2}$ $s=0$ par kissing karte hue — shaded zoom window mein indistinguishable. Woh kiss hi reason hai ki sirf variance matter karta hai.

---

Step 7 — Sum karo, factor karo, aur limit lo → bell ka fingerprint

Ab assemble karo. Sum ko standardize karne par $Z_n=\frac{1}{\sqrt{n}}{\sum }_{i=1}^n{Y}_i$ milta hai. Multiply-fingerprints rule apply karo:

${\phi }_{Z_n}(t)=\left[{\phi }_Y\right(\frac{t}{\sqrt{n}}){]}^n={\left(1-\frac{t^2}{2n}+o\left(\frac{1}{n}\right)\right)}^n\underset{n\to \infty }{\to }{e}^{-t^2/2}.$

• Humne Step 6 mein $s=\frac{t}{\sqrt{n}}$ plug kiya, isliye $\frac{s^2}{2}=\frac{t^2}{2n}$.
• Power $n$ aata hai $n$ identical independent factors multiply karne se.
• Limit use karta hai ${\left(1+\frac{a}{n}\right)}^n\to e^a$ with $a=-\frac{t^2}{2}$ — wahi limit jo compound interest define karta hai.

Aur $e^{-t^2/2}$ exactly standard normal $N(0,1)$ ka fingerprint hai. Fingerprints match ⟹ distributions match (Lévy's continuity theorem). Ho gaya. $■$

KAISA DIKHTA HAI: curves ${\left(1-\frac{t^2}{2n}\right)}^n$ for $n=1,4,20,\infty$ smooth red target $e^{-t^2/2}$ par climb karte hue. Jaise jaise $n$ badhta hai woh bell ke fingerprint par lock ho jaate hain.

---

Step 8 — Woh edge case jo machine ko TODTA hai

Upar ke har step ko finite variance ${\sigma }^2<\infty$ chahiye tha. Agar ek distribution itni heavy-tailed ho ki ${\sigma }^2=\infty$?

KAISA DIKHTA HAI: side by side, exponential ke averages ek bell mein tight hote hain (green) jabki Cauchy ke averages bade $n$ par bhi wild aur spread out rehte hain (red). Ek occasional monster draw poore average ko drag karta hai.

---

Ek-picture summary

Left se right: koi bhi starting shape → bohot saare size-$n$ averages lo → $\sigma /\sqrt{n}$ se center aur rescale karo → piled, standardized averages ek universal bell $N(0,1)$ par converge karte hain, jiska fingerprint $e^{-t^2/2}$ hai — jab tak variance infinite na ho, us case mein (Cauchy) machine jam ho jaati hai.

Recall Poore walkthrough ki Feynman retelling

Koi bhi messy random cheez lo — ek dice, ek lopsided waiting time, ek coin flip. Ek draw unpredictable hai (Step 0–1). Toh ek draw predict mat karo; bohot saare ko average karo (Step 1). Woh averaging baar baar karo aur answers pile up karo — woh pile ek brand-new picture hai (Step 2). Pile hamesha true mean par baithti hai, aur woh one-over-root-n ki tarah narrow hoti hai (Step 3). Har pile ko ek hi size par shrink aur shift karo taaki shapes compare kar sakein (Step 4). Shape prove karne ke liye, us "fingerprint" par switch karo jo randoms add karne ko numbers multiply karne mein badal deta hai (Step 5), zoom in karo taki sirf spread matter kare (Step 6), $n$ identical fingerprints multiply karo aur limit lo — $e^{-t^2/2}$ nikalta hai, bell ka fingerprint (Step 7). Yeh kisi bhi finite width wali cheez ke liye kaam karta hai, aur dramatically fail karta hai fat-tailed Cauchy ke liye (Step 8). Yahi puri magic hai: mess in, bell out.

Recall Quick self-checks

Deviation ko $\sqrt{n}$ se divide kyun karte hain aur $n$ se nahi? ::: $n$ se divide karna over-squash kar ke $0$ kar deta hai (Law of Large Numbers); $\sqrt{n}$ se width ko standard error par freeze karta hai taki ek non-degenerate bell survive kare. Kya raw data normal ban jaata hai? ::: Nahi — sirf mean/sum ki sampling distribution hoti hai; raw data apni shape rakhta hai. Characteristic functions proof ko easy kyun banate hain? ::: Independent sums apne fingerprints multiply karte hain, aur pointwise CF convergence convergence in distribution imply karta hai. Limit $X$ ke kaun se do facts par depend karta hai? ::: Sirf uska finite mean $\mu$ aur finite variance ${\sigma }^2$. CLT Cauchy ke liye fail kyun hota hai? ::: Uski variance infinite hai, isliye rescale karne ke liye koi finite width nahi aur Taylor $-t^2/2$ term kabhi appear nahi hota.

---

See also

Confidence Intervals · Hypothesis Testing — z and t tests · Binomial Distribution · Standard Error · Law of Large Numbers