Visual walkthrough — Central limit theorem

1.3.15 · D2 · AI-ML › Probability & Statistics › Central limit theorem

Pehli line se pehle, teen simple vaade:

• Ek random variable bas "ek aisa number hai jo aapko abhi tak nahi pata jab tak dekhte nahi" — jaise die ka face girane se pehle. Hum ise capital letter se likhte hain, jaise $X$.
• Mean $\mu$ (Greek letter "mu") $X$ ka long-run average hai — uski picture ka balance point.
• Variance ${\sigma }^2$ ("sigma squared") spread measure karta hai — average mein values balance point se kitni door hain, squared. Iska square root $\sigma$ ek aam "center se typical distance" hai.

Baaki sab hum aage earn karenge.

---

Step 1 — KISI BHI shape se shuru karo

KYA. Ek single random variable $X$ lo. Hum yeh ASSUME nahi karte ki yeh bell hai. Figure dekho: left mein ek flat uniform die hai (chhe equal bars), beech mein ek tedha skewed distribution hai, right mein ek jagged two-spike wala. Har ek ka balance point $\mu$ ek vertical line se mark hai, aur typical spread $\sigma$ ek chhote arrow se.

KYU ugly se shuru karo. CLT ka poora miracle yeh hai ki shuru ki shape matter nahi karti. Agar hum sirf un shapes ke liye prove karte jo pehle se bells jaisi dikhti hain, toh hum kuch prove nahi karte. Isliye hum jaanbujhkar sabse buri dikhne wali distributions se shuru karte hain.

PICTURE.

Yahan har symbol:

• $X$ — source se ek draw (ek die roll, ek data point).
• $\mu$ — vertical balance line. Teeno shapes mein same idea.
• $\sigma$ — horizontal spread arrow; lamba arrow = zyada spread.

Ab se sab yahi hai: agar hum is $X$ ke kaafi saare independent copies average karein toh kya hoga?

---

Step 2 — Averaging machine

KYA. $n$ independent copies lo $X_1,X_2,\dots ,X_n$ — sabhi same ugly shape se drawn (yeh i.i.d. assumption hai: independent, identically distributed). Inhe ek aisi machine mein daalo jo sum kare aur divide kare: ${\bar{X}}_n=\underset{\text{count se divide karo}}{\underbrace{\frac{1}{n}}}\underset{\text{sab add karo}}{\underbrace{\sum _{i=1}^n{X}_i}}$

Term by term: ${\bar{X}}_n$ (padho "X-bar") sample mean hai — is batch ka average. $\sum$ ("sigma", Sum ke liye capital S) bas kehta hai "$X_1$ se $X_n$ tak add karo." $\frac{1}{n}$ total ko average mein badalta hai.

KYU sirf sum nahi, average karo. Ek raw sum bina limit ke badhta rehta hai jaise $n$ badhta hai, toh uski picture hamesha right ki taraf drift karti rehti hai. $n$ se divide karne par picture $\mu$ ke aaspaas pin ho jaati hai taaki hum uski shape study kar sakein, na ki uski runaway position.

PICTURE. Funnel kaafi draws leti hai, aur ek number ${\bar{X}}_n$ emit karti hai.

${\bar{X}}_n$ khud random hai — machine dobara chalao, thoda alag average milega. Toh ${\bar{X}}_n$ ki apni distribution hai, apni picture. Woh picture dhundhna hi poora game hai.

---

Step 3 — Average kahan baithta hai, aur kitna tight?

KYA. Machine ke output ke baare mein do facts, pehle ki hum uski shape poochhen.

Uska center: $\mathbb{E}[{\bar{X}}_n]=\mu$ Averaging machine balance point nahi hilati. ($\mathbb{E}[\cdot ]$ ka matlab "expected value" = long-run average.)

Uski spread:

\qquad\Longrightarrow\qquad \underbrace{\text{SE}}_{\text{standard error}} \;=\; \frac{\sigma}{\sqrt{n}}$$ Term by term: $\sigma^2$ *ek* draw ki spread thi. $n$ se divide karo toh yeh shrink hoti hai. Square root lo taaki wapas everyday distance units mein aao: **standard error** $\text{SE} = \sigma/\sqrt{n}$ — $\bar{X}_n$ ki $\mu$ se typical distance. **KYU $\sigma^2/n$ — pictures mein bana ke dekho.** Yeh load-bearing formula hai, toh isse *dekhte* hain, bas state nahi karte. Figure mein teen panels dekho. *Panel A — independent cheezein ke liye variances add hote hain.* Spread ko ek stick ki length ki taah socho jo random direction mein point karti hai. Jab do independent random draws add hote hain, unki sticks *unrelated* directions mein point karti hain, toh woh right triangle ke do legs ki tarah combine hote hain: naya spread-squared $\sigma^2 + \sigma^2$ hai, *squares* ka sum (Pythagoras!). Yeh exactly woh rule hai $\text{Var}(X_1+X_2)=\sigma^2+\sigma^2$ independent draws ke liye. $n$ baar karo: $$\text{Var}\!\left(\sum_{i=1}^n X_i\right)=\underbrace{\sigma^2+\sigma^2+\cdots+\sigma^2}_{n\text{ terms}}=n\sigma^2$$ Har $\sigma^2$ ek draw ki spread hai; woh add hote hain kyunki independence unke wiggles ko align hone se rokta hai. *Panel B — $n$ se divide karna axis ko squeeze karta hai.* Sample mean woh sum hai $n$ se divided. Picture ko horizontally $n$ factor se squeeze karne par distances $n$ se divide ho jaati hain, isliye spread-*squared* $n^2$ se (factor ko square karke): $$\text{Var}(\bar{X}_n)=\text{Var}\!\left(\tfrac{1}{n}\textstyle\sum X_i\right)=\frac{1}{n^2}\,(n\sigma^2)=\frac{\sigma^2}{n}$$ Yahan $1/n^2$ squeeze factor squared hai, $n\sigma^2$ Panel A ka summed spread hai; woh cancel ho ke $\sigma^2/n$ banta hai. *Panel C — $\sqrt{n}$ punchline.* Plain distance mein wapas aane ke liye square root lo: $\text{SE}=\sigma/\sqrt{n}$. Isliye sample size chaar guna karne par error sirf *aadha* hota hai. **PICTURE.** ![[deepdives/dd-ai-ml-1.3.15-d2-s03.png]] Jaise $n$ badhta hai, $\bar{X}_n$ ki picture taller aur narrower hoti jaati hai, hamesha $\mu$ par centered. Lekin "narrow" ka matlab "bell-shaped" nahi hai. Ek narrow triangle abhi bhi triangle hai. Step 5 dikhata hai kyun yeh *zaroor* bell banna chahiye. > [!mistake] LLN aur CLT do alag guarantees hain > Shrinking $\text{SE}=\sigma/\sqrt{n}\to 0$ kehta hai average $\mu$ par *home in* karta hai — woh akele [[Law of Large Numbers]] (LLN) hai: average *kahan* girta hai. CLT ek *alag, tezz* statement hai **kis shape** ka wobble $\mu$ ke aaspaas hota hai jab aap zoom in karte ho — ek bell. LLN wobble ko khatam karta hai; CLT wobble ki shape *describe* karta hai *pehle* ki woh marta hai. Inhe confuse mat karo: LLN ko sirf finite mean chahiye, CLT ko finite variance bhi chahiye. --- ## Step 4 — Rescale karo taaki picture shrink hona band kare **KYA.** Agar hum $\bar{X}_n$ ko ek spike mein collapse hone dein, uski shape ek point mein kho jaayegi. Toh hum **standardize** karte hain — center subtract karo, spread se divide karo — taaki picture unit width par freeze ho jaaye: $$Z_n \;=\; \frac{\overbrace{\bar{X}_n - \mu}^{\text{0 par center karne ke liye shift karo}}}{\underbrace{\sigma/\sqrt{n}}_{\text{width 1 ke liye stretch karo}}}$$ Term by term: $\mu$ subtract karna balance point ko $0$ par slide karta hai; $\text{SE}=\sigma/\sqrt{n}$ se divide karna horizontal axis ko stretch karta hai taaki spread exactly $1$ ho jaaye. Ab $Z_n$ ka mean $0$ aur variance $1$ hai *har* $n$ ke liye. **KYU exactly yeh transform.** Hume alag $n$ ke across fair side-by-side comparison chahiye. Do distributions jinke alag centers aur widths hain unhe aankhon se compare nahi kar sakte. Standardizing boring differences (position, scale) hatata hai taaki sirf *shape* baaki rahe examine karne ke liye. **PICTURE.** Same underlying distribution, $0$ par re-centered aur width $1$ par re-scaled — ek moving camera jo shrinking blob ko track karti hai aur use frame mein rakhti hai. ![[deepdives/dd-ai-ml-1.3.15-d2-s04.png]] Yeh same standardization hai jo [[Normal Distribution]] aur [[Confidence Intervals]] banane mein use hoti hai. --- ## Step 5 — Shape converge hote dekho (miracle) **KYA.** Ab camera $Z_n$ par rakho aur $n$ badhao: $n=1$, phir $2$, phir $5$, phir $30$. True bell curve — likha $\mathcal{N}(0,1)$ — dashed line mein overlay karo. Pehle, $\mathcal{N}(0,1)$ ka *matlab* kya hai? Letter $\mathcal{N}$ bas shorthand hai "**N**ormal" ke liye, perfect bell curve ka technical naam (iska full formula [[Normal Distribution]] mein hai). Brackets mein do numbers uska center aur spread hain: $\mathcal{N}(0,1)$ = "ek bell $0$ par centered jiska variance $1$ hai." Toh $\mathcal{N}(0,1)$ simply *standard bell* hai — isse zyada kuch nahi. Flat die se shuru karke, dekho: - $n=1$: flat (die khud). - $n=2$: ek triangle (do dice sum karna — tent shape). - $n=5$: already rounded aur humped. - $n=30$: dashed bell se indistinguishable. **KYU hota hai — intuition.** Independent shapes ko baar baar convolve (add) karne par har bump *average out* hota hai. Sharp corners ke liye parts ke beech fine coordination chahiye; independence woh coordination forbid karti hai, isliye corners sand down hote hain. Ek hi shape hai jo is endless smoothing ke under stable hai — jo shape khud ko reproduce karti hai jab aap do copies add karte ho — woh bell hai. Yeh averaging machine ka "fixed point" hai, isliye sab kuch uski taraf flow karta hai. **PICTURE.** ![[deepdives/dd-ai-ml-1.3.15-d2-s05.png]] > [!formula] Picture kis ki taraf converge ho rahi hai > $$Z_n \;=\; \frac{\bar{X}_n - \mu}{\sigma/\sqrt{n}} \;\xrightarrow{\;d\;}\; \mathcal{N}(0,1)$$ > Arrow $\xrightarrow{d}$ ka matlab hai "picture (distribution) converge hoti hai," na ki individual numbers converge karte hain. $\mathcal{N}(0,1)$ standard bell hai jo abhi define ki: center $0$, spread $1$. --- ## Step 6 — Kyun bell aur kuch nahi (self-reproducing shape) **KYA.** Do independent bells lo, unhe add karo, standardize karo — phir se bell milti hai. Ek triangle ko ek triangle mein add karo toh kuch rounder milta hai, *triangle nahi*. Sirf bell addition se reproduce hoti hai. **KYU yeh confirm karta hai.** Step 5 ne kaha "sab kuch ek stable shape ki taraf flow karta hai." Step 6 woh shape uniquely identify karta hai: finite variance wali saari distributions mein, Gaussian *akela* hai jo (standardized) addition ke under closed hai. Toh Step 5 mein limit kuch aur nahi ho sakti. Yeh purely ek picture-argument hai: baar baar add-and-smooth karne par ek aisi shape milni chahiye jo add-and-smooth karne ke baad dobara nahi badalti, aur bell hi aisi akeli shape hai. **PICTURE.** Do bells → ek bell (top row, stable). Do triangles → ek hump (bottom row, changed). ![[deepdives/dd-ai-ml-1.3.15-d2-s06.png]] > [!recall]- Algebra lovers ke liye: main note kaisa nail karta hai > Upar ka picture-argument result *dekhne* ke liye kaafi hai. Agar aap watertight proof chahte ho, [[Central limit theorem|main note]] ek tool use karta hai jise **moment generating function (MGF)** kehte hain: ek tarika poori distribution ko ek single function $M(t)=\mathbb{E}[e^{tX}]$ ki tarah summarize karne ka, jaise uski shape ka barcode. Same barcode wali do distributions same distribution hoti hain. Proof dikhata hai ki $Z_n$ ka barcode $e^{t^2/2}$ ki taraf tend karta hai, jo precisely $\mathcal{N}(0,1)$ ka barcode hai — toh limit *hai* bell. Aapko pictures par trust karne ke liye yeh machinery nahi chahiye; yeh bas "bell ki taraf flow karta hai" ko mathematically exact banata hai. Key algebraic fact yeh hai ki sirf $X$ ka mean aur variance $t/\sqrt{n}$ scaling ke baad bachte hain — har shape-specific detail wash out hoti hai, jo exactly isliye hai ki starting shape bhool jaati hai. --- ## Step 7 — Edge cases (jahan machine toot ti ya ruk jaati hai) **KYA.** Char boundary scenarios, figure mein har ek ka apna panel: 1. **$n=1$ (abhi averaging nahi).** $Z_1 = (X-\mu)/\sigma$ bas aapki original ugly shape hai, standardized. Koi bell nahi. Theorem ek *limit* hai — chhote $n$ ke baare mein yeh kuch nahi kehta. 2. **Skew convergence slow karta hai — aur kitna.** Start jitna rough (zyada lopsided), utna bada $n$ chahiye pehle bell appear karne se. Iske liye ek concrete dial hai: **skewness** $\gamma$ lopsidedness measure karta hai (ek symmetric shape ka $\gamma=0$ hota hai; exponential shape ka $\gamma=2$). Bell approximation ki leading error $\gamma/\sqrt{n}$ ki tarah shrink karti hai (yeh pehla Edgeworth-correction term hai). Toh error aadha karne ke liye $n$ *chaar guna* karna padta hai, aur double skew wale source ko same accuracy ke liye chaar guna samples chahiye. Isliye ek symmetric die $n=5$ tak nearly bell hoti hai, jabki sharply skewed source ko $n$ hundreds mein chahiye. 3. **Infinite variance ($\sigma^2=\infty$).** Machine **toot jaati hai**. Heavy-tailed sources (jaise Cauchy) assumption $\sigma^2<\infty$ violate karte hain; unke averages normal *nahi* bante. Poori derivation ne Step 4 mein $\sigma$ use kiya — $\sigma$ se divide undefined hai agar $\sigma$ infinite ho. 4. **Degenerate ($\sigma=0$).** Agar $X$ constant hai, koi randomness nahi: $\bar{X}_n=\mu$ hamesha, ek single spike. Step 4 $0$ se divide karta hai — undefined. Koi distribution nahi converge karne ke liye. **KYU yeh dikhate hain.** Ek theorem tabhi trustworthy hai jab aap exactly jaano ki woh kab *fail* karta hai. Dono failure modes Step 4 ke denominator $\sigma$ par attack karte hain: $\sigma=\infty$ aur $\sigma=0$. Us range mein strictly between $(0,\infty)$ sab kuch kaam karta hai; us range ke andar *speed* skewness $\gamma$ se governed hai. **PICTURE.** Char panels: works-eventually ($n=1$ raw), works-slowly (skewed, apni $\gamma/\sqrt{n}$ error ke saath shrinking), breaks (heavy tail), degenerate spike. ![[deepdives/dd-ai-ml-1.3.15-d2-s07.png]] > [!mistake] Do classic traps > - "n=30 hamesha normality guarantee karta hai." Galat — heavy skew ya heavy tails ko bahut zyada chahiye. Kyunki error $\gamma/\sqrt{n}$ ki tarah girta hai, bada $\gamma$ bada $n$ demand karta hai. 30 wala rule folklore hai, theorem nahi. > - "CLT mera *data* normal banata hai." Nahi. Yeh **sample mean** ki distribution ko normal banata hai. Individual data utna hi ugly rehta hai jitna tha. --- ## Ek-picture summary ![[deepdives/dd-ai-ml-1.3.15-d2-s08.png]] Left se right padho: koi bhi ugly shape → $n$ inhe average karo → center aur rescale karo → aur chahe aap kahan se bhi shuru karo, picture same bell par land karti hai. Arrow par teen conditions hi poora fine print hain: **i.i.d.**, **finite variance**, aur **$n$ itna bada** (kitna bada yeh skewness $\gamma$ par depend karta hai). > [!recall]- Feynman retelling — plain words mein kaho > Socho ek machine hai jo *kisi bhi* junky distribution se $n$ random numbers uthati hai, unhe add karti hai, aur $n$ se divide karti hai. Woh output — sample mean — true average $\mu$ se bahut door nahi jaata, aur jaise zyada numbers use karte ho yeh tighter hota jaata hai, $n$ ke square root ki tarah tight. Kyun $\sqrt{n}$? Kyunki independent spreads right triangle ke legs ki tarah add hote hain (spread-squared add hota hai), aur phir $n$ se divide karne par woh wapas neeche squeeze hota hai — do effects mila ke $\sigma/\sqrt{n}$ rehta hai. Agar ab us shrinking blob par ek moving camera rakho — ise zero par slide karo aur zoom karo taaki width hamesha ek rahe — toh aap uski shape ko ek smooth bell mein freeze hote dekhte ho, exactly standard normal $\mathcal{N}(0,1)$. Kyun bell aur kuch nahi? Kyunki independent randomness add karna har corner ko sand karta hai, aur bell woh ek shape hai jo khud mein add hone ke baad survive karti hai. Trick sirf do jagah fail hoti hai: agar source ki spread infinite ho (divide karne ke liye kuch nahi, tails jeet jaate hain), ya zero spread (ek dead constant, koi shape hi nahi). Un dono ke beech sab jagah, bell destiny hai — halaanki lopsided (skewed) start aapko zyada wait karwata hai, error sirf $\gamma/\sqrt{n}$ ki tarah dying. Aur do facts confuse mat karo: *kahan* average land karta hai woh Law of Large Numbers hai; *kis shape* ka uska wobble hai woh CLT hai. > [!recall]- Quick self-check > Sum ko $n$ se kyun divide karo sirf sum karne ki jagah? ::: Raw sum infinity ki taraf drift karta hai jaise $n$ badhta hai; $n$ se divide karna picture ko $\mu$ ke aaspaas pin karta hai taaki hum uski shape study kar sakein. > Standard error $\sqrt{n}$ ki tarah kyun shrink karta hai, $n$ ki tarah kyun nahi? ::: Independent spreads right triangle ke legs ki tarah add hote hain ($\sigma^2+\sigma^2+\cdots=n\sigma^2$), phir $n$ se divide karne par variance $1/n^2$ se scale hota hai, $\sigma^2/n$ bachta hai; square root $\sigma/\sqrt{n}$ deta hai. > Standardizing (subtract $\mu$, SE se divide karo) kya accomplish karta hai? ::: Yeh center ko $0$ par slide karta hai aur width ko $1$ tak stretch karta hai, taaki sirf shape baaki rahe compare karne ke liye alag $n$ ke across. > LLN aur CLT mein kya difference hai? ::: LLN kehta hai average *kahan* land karta hai (woh $\mu$ par home in karta hai); CLT kehta hai $\mu$ ke aaspaas wobble ki *shape* kya hai (ek bell). LLN ko finite mean chahiye, CLT ko finite variance. > CLT ke do failure modes naam batao. ::: Infinite variance ($\sigma^2=\infty$, heavy tails) aur degenerate zero variance ($\sigma=0$, ek constant). Dono divide-by-$\sigma$ step ko khatam karte hain. > Skewness convergence speed ko kaise affect karta hai? ::: Bell-approximation error $\gamma/\sqrt{n}$ ki tarah girta hai, toh zyada skewed source (bada $\gamma$) ko same accuracy ke liye bada $n$ chahiye. --- **Yeh bhi dekho:** [[Bootstrap Methods]] (resampling same sampling-distribution idea par lean karta hai), [[Sample Size Calculation]] ($n$ choose karne ke liye $\sigma/\sqrt{n}$ shrink use karta hai), [[Monte Carlo Methods]] (integrals estimate karne ke liye random draws average karna), [[Maximum Likelihood Estimation]] aur [[Bias-Variance Tradeoff]] iske liye ki estimators kaise behave karte hain jab normality kick in kare.