Visual walkthrough — Monte Carlo simulation — law of large numbers basis

4.9.25 · D2 · Maths › Probability Theory & Statistics › Monte Carlo simulation — law of large numbers basis

Characters, koi bhi maths aane se pehle

Main har symbol ko introduce kar deta hoon taaki kuch bhi achanak na aaye.

Step 1 — Ek random draw kaisa dikhta hai

KYA. Kuch bhi average karne se pehle, ek single draw $X_i$ dekho. Yeh ek dot hai jo kahin land karti hai, true centre $\mu$ ke aaspaas scatter hokar.

KYUN. Aap bahut saari cheezein ka average tab tak nahi samajh sakte jab tak ek cheez na dekho. Us ek dot ka spread exactly wahi hai jo $\sigma$ measure karta hai — $\mu$ se typical distance.

PICTURE. Neeche, yellow line true mean $\mu$ hai. Blue dots individual draws $X_i$ hain; woh har jagah land hoti hain, kabhi kabhi $\mu$ se bahut door. Red band ek $\sigma$ ka typical spread dikhata hai.

$\underset{\text{one draw}}{\underbrace{X_i}}\approx \underset{\text{true centre}}{\underbrace{\mu }}\pm \underset{\text{typical wobble}}{\underbrace{\sigma }}$

Har single draw noisy hai. Ek single poke aapko almost kuch nahi batati. Magic averaging mein hai.

Step 2 — Average exactly $\mu$ pe baithta hai (unbiased)

KYA. Hum sample mean ki expected value compute karte hain: average pe, ${\bar{X}}_n$ kahan centred hai?

KYUN. Ek estimator tabhi trustworthy hai jab woh systematically zyada high ya zyada low aim na kare. Hume confirm karna hoga ki woh spread ki chinta karne se pehle sahi target pe point karta hai.

$\mathbb{E}[{\bar{X}}_n]=\mathbb{E}\left[\underset{\text{constant}}{\underbrace{\frac{1}{n}}}\underset{\text{sum of draws}}{\underbrace{{\sum }_{i=1}^n{X}_i}}\right]=\frac{1}{n}{\sum }_{i=1}^n\underset{=\mu \text{ each}}{\underbrace{\mathbb{E}[X_i]}}=\frac{1}{n}\cdot n\mu =\mu .$

Term by term: $\frac{1}{n}$ ek constant hai, isliye woh expectation se bahar slide kar jaata hai. Linearity of expectation $\mathbb{E}$ ko sum ke through pass hone deta hai. $n$ pieces mein se har ek $\mu$ ke barabar hai, jisse $n\mu$ milta hai, aur $n$s cancel ho jaate hain.

PICTURE. Blue dots scatter karti hain, lekin unka balance-point (green marker, yani average) yellow $\mu$ line pe seedha baitha hai — chahe dots kitni bhi kam ya zyada hon.

Step 3 — Averaging spread ko $1/n$ se shrink karta hai (variance mein)

KYA. Hum sample mean ka variance compute karte hain — ${\bar{X}}_n$ itself typically $\mu$ se kitna door bhatakta hai.

KYUN. Sirf unbiased hona kaafi nahi: ek dart-thrower jo bullseye pe centred ho lekin har jagah wild ho, useless hai. Hume prove karna hai ki ${\bar{X}}_n$ ki wildness $n$ badhne ke saath girti hai.

$\mathrm{Var}({\bar{X}}_n)=\mathrm{Var}(\underset{\text{pull out as }1/n^2}{\underbrace{\frac{1}{n}}}\sum X_i)=\frac{1}{n^2}\underset{\text{independence: no cross terms}}{\underbrace{\sum _{i=1}^n\mathrm{Var}(X_i)}}=\frac{1}{n^2}\cdot n{\sigma }^2=\frac{{\sigma }^2}{n}.$

Term by term: constant $\frac{1}{n}$ ko variance se bahar nikalte hain toh woh square ho jaata hai (rule: $\mathrm{Var}(aY)=a^2\mathrm{Var}(Y)$), isliye woh $\frac{1}{n^2}$ ban jaata hai. Independence hi hai jo saare cross-covariance terms ko khatam karta hai — iske bina yeh step galat hai (Step 6 mein degenerate case dekho). Har $\mathrm{Var}(X_i)={\sigma }^2$ hai, aur $n$ of them $n{\sigma }^2$ dete hain. Neeche ka $n^2$ upar ke single $n$ se zyada powerful hai, aur ${\sigma }^2/n$ bachta hai.

PICTURE. Average dot ke teen stacked clouds: $n=1$, $n=10$, $n=100$ ke liye. Jaise $n$ badhta hai, averages ka cloud $\mu$ ki taraf collapse karta hai.

Step 4 — Variance se ek guarantee tak: Chebyshev's inequality

KYA. Hum "small variance" ko probability ke baare mein ek hard statement mein convert karte hain: ${\bar{X}}_n$ kitni baar $\mu$ se door land ho sakta hai?

KYUN. Variance squared distances ka average hai — helpful, lekin hume ek waada chahiye jaise "itna door hone ka chance $\epsilon$ se zyada nahi hai." Chebyshev's Inequality exactly variance se bade miss ki probability ka bridge hai, aur ise sirf variance ki zaroorat hai.

$P\left(\underset{\text{miss by at least }\epsilon }{\underbrace{|{\bar{X}}_n-\mu |\ge \epsilon }}\right)\le \frac{\mathrm{Var}({\bar{X}}_n)}{{\epsilon }^2}=\frac{{\sigma }^2}{n{\epsilon }^2}.$

Term by term: $\epsilon$ (Greek "epsilon") ek tolerance hai jo hum choose karte hain — "kitna close kaafi close hai". Left side uss amount ya zyada se miss hone ki probability hai. Chebyshev use variance divided by ${\epsilon }^2$ se cap karta hai. Step 3 ka ${\sigma }^2/n$ substitute karne se final bound milta hai.

PICTURE. Possible ${\bar{X}}_n$ values ki bell, jisme green mein tolerance band $\mu \pm \epsilon$ shaded hai. Chebyshev band ke bahar red tail area ko ${\sigma }^2/(n{\epsilon }^2)$ se bound karta hai.

Step 5 — $n\to \infty$ karte hain: tail zero pe crush ho jaata hai

KYA. $\epsilon$ ko fixed rakho aur sample size ko bina bound ke badhne do.

KYUN. Yeh finish line hai. Agar bound khud $0$ pe jaata hai, toh probability jo use cap karti hai woh bhi $0$ pe squeeze ho jaati hai — yahi convergence hai.

${\lim }_{n\to \infty }P(|{\bar{X}}_n-\mu |\ge \epsilon )\le {\lim }_{n\to \infty }\underset{\text{numerator fixed, denom }\to \infty }{\underbrace{\frac{{\sigma }^2}{n{\epsilon }^2}}}=0.$

Term by term: ${\sigma }^2$ aur ${\epsilon }^2$ frozen constants hain; sirf denominator mein $n$ badhta hai. Kuch fixed upar aur neeche kuch infinity ki taraf jaata ho toh result $0$ hota hai. Kyunki probability negative nahi ho sakti, woh exactly $0$ pe trap ho jaati hai. Yahi Weak Law of Large Numbers hai: ${\bar{X}}_n\stackrel{P}{\to }\mu$. $■$

PICTURE. Bound ${\sigma }^2/(n{\epsilon }^2)$ ko $n$ ke against plot kiya — ek curve axis ki taraf dive karti hai. Upar overlay: ${\bar{X}}_n$ ki bell $\mu$ pe spike mein narrow hoti hui.

Step 6 — Degenerate case: guarantee kyun toot jaati hai

KYA. Upar har step secretly do cheezein maanta tha: independence aur finite variance. Dekho kya hota hai jab koi bhi fail ho.

KYUN. Contract kehta hai har case cover karo. Ek reader jo correlated samples ya heavy-tailed integrand use kare aur yeh kabhi na dekhe, woh blindsided ho jaayega.

• Correlated draws. Step 3 mein, independence ne cross-terms delete kiye. Agar draws correlated hain, extra covariance terms bachte hain, aur variance ${\sigma }^2/n$ se zyada hota hai — kabhi kabhi woh kabhi shrink hi nahi karta. Averaging se barely madad milti hai.
• Infinite variance. Step 4 mein, Chebyshev ko finite ${\sigma }^2$ chahiye tha. Heavy-tailed variable ke liye (jaise Cauchy-like $g$), ${\sigma }^2=\infty$ hai aur bound ${\sigma }^2/(n{\epsilon }^2)$ useless hai. Sample mean forever wander kar sakta hai aur kabhi settle nahi hoga.

PICTURE. Do running-average traces vs $n$. Green: independent, finite variance — smoothly $\mu$ pe converge karta hai. Red: heavy-tailed — jumps karta hai aur kabhi settle nahi hota.

Ek-picture summary

Sab kuch ek canvas pe: single noisy draws (Step 1) → unka average $\mu$ pe land karta hai (Step 2) → averaging spread ko ${\sigma }^2/n$ ki tarah tighten karta hai (Step 3) → Chebyshev tail ko cap karta hai (Step 4) → $n\to \infty$ use zero pe crush karta hai (Step 5), sivaaye jab independence ya finite variance fail ho (Step 6).

Active Recall