Law of Large Numbers — weak and strong

4.9.16 · Maths › Probability Theory & Statistics

HUM ACTUALLY KYA CLAIM KAR RAHE HAIN?

Maano $X_1,X_2,\dots$ i.i.d. (independent, identically distributed) random variables hain jinka finite mean $\mu =\mathbb{E}[X_i]$ hai. Sample mean define karo:

${\bar{X}}_n=\frac{1}{n}{\sum }_{i=1}^n{X}_i$

"${\bar{X}}_n\to \mu$" ke do flavours hain, kyunki random variables ki sequence converge hone ke do tarike hote hain.

WEAK LAW DERIVE KAISE KAREIN (scratch se)

Hum ise do elementary inequalities se banate hain. Inhe bhi derive karo.

Step 1 — Markov's inequality

Ek non-negative random variable $Y\ge 0$ aur $a>0$ ke liye: $\mathbb{E}[Y]={\int }_0^{\infty }yf(y)dy\ge {\int }_a^{\infty }yf(y)dy\ge {\int }_a^{\infty }af(y)dy=a\mathbb{P}(Y\ge a)$ Har step kyun? Pehla $\ge$: humne $[0,a)$ waala part throw away kar diya (non-negative hai, sirf shrink hoga). Doosra $\ge$: $[a,\infty )$ par $y\ge a$ hota hai. Rearrange karo: $\boxed{\mathbb{P}(Y\ge a)\le \frac{\mathbb{E}[Y]}{a}}$

Step 2 — Chebyshev's inequality

Markov ko $Y=(X-\mu {)}^2\ge 0$ par apply karo jahan $a={\epsilon }^2$ hai: $\mathbb{P}((X-\mu {)}^2\ge {\epsilon }^2)\le \frac{\mathbb{E}[(X-\mu {)}^2]}{{\epsilon }^2}=\frac{{\sigma }^2}{{\epsilon }^2}$ Kyunki $(X-\mu {)}^2\ge {\epsilon }^2⟺|X-\mu |\ge \epsilon$: $\boxed{\mathbb{P}(|X-\mu |\ge \epsilon )\le \frac{{\sigma }^2}{{\epsilon }^2}}$

Step 3 — Sample mean ka Variance

Yeh step kyun? Chebyshev ko ${\bar{X}}_n$ ka variance chahiye. Independence use karte hue (covariances vanish ho jaate hain) aur identical distribution se: $\mathrm{Var}({\bar{X}}_n)=\mathrm{Var}(\frac{1}{n}\sum X_i)=\frac{1}{n^2}{\sum }_{i=1}^n\mathrm{Var}(X_i)=\frac{n{\sigma }^2}{n^2}=\frac{{\sigma }^2}{n}$ Saath hi $\mathbb{E}[{\bar{X}}_n]=\mu$ (linearity se). Variance $1/n$ ki tarah shrink karta hai — yahi engine hai.

Step 4 — Sab ek saath lagao

${\bar{X}}_n$ par Chebyshev apply karo (mean $\mu$, variance ${\sigma }^2/n$): $\mathbb{P}(|{\bar{X}}_n-\mu |\ge \epsilon )\le \frac{{\sigma }^2/n}{{\epsilon }^2}=\frac{{\sigma }^2}{n{\epsilon }^2}\underset{n\to \infty }{\to }0$

STRONG LAW KAISE KAAM KARTA HAI (idea, poora epsilon-grind nahi)

SLLN (Kolmogorov) ko sirf $\mathbb{E}|X_i|<\infty$ chahiye. Key tool hai Borel–Cantelli lemma:

$A_n=\{|{\bar{X}}_n-\mu |>\epsilon \}$ lo. Crude Chebyshev bound ${\sum }_n\frac{{\sigma }^2}{n{\epsilon }^2}$ deta hai jo diverge karta hai — isliye naive bound kaafi nahi hai. Yahi wajah hai ki SLLN, WLLN se zyada mushkil hai. Kolmogorov ek sharper maximal inequality use karta hai (ek saath $n$ tak worst deviation control karta hai) taaki relevant series converge kare, phir Borel–Cantelli saare finitely many bure events ko khatam karta hai $\Rightarrow$ path hamesha ke liye settle ho jaata hai.

Worked Examples

Common Mistakes (Steel-manned)

Active Recall

Flashcards

Connections

• Chebyshev's Inequality — WLLN ki rate ke liye workhorse.
• Markov's Inequality — Chebyshev ka parent.
• Central Limit Theorem — $1/\sqrt{n}$ fluctuations describe karta hai jo LLN ignore karta hai.
• Borel–Cantelli Lemma — Strong Law ka engine.
• Modes of Convergence — in probability vs almost sure vs in distribution.
• Monte Carlo Methods — LLN ka practical payoff.
• Gambler's Fallacy — LLN ki famous galat reading.

Concept Map

Deep Dive

• D1 · Foundations — har symbol zero se
• D2 · Visual walkthrough — pictures mein derivation
• D3 · Worked examples — har scenario
• D4 · Exercises — graded, full solutions
• D5 · Question bank — concept traps