Confidence intervals

1.3.21 · AI-ML › Probability & Statistics

What Problem Does This Solve?

Point estimates jhooth bolte hain. Aap sample mean $\bar{x}=170$ cm calculate karte ho, lekin yeh sirf ek random process ka ek realization hai. Confidence interval quantify karta hai ki hamen is estimate par kitna trust karna chahiye.

ML mein yeh kyun zaroori hai?

• Model accuracy report karna: "Test accuracy hai 87% ± 2%"
• A/B testing: "Treatment ne conversion 3.2% badhaya [2.1%, 4.3%]"
• Hyperparameter tuning: cross-validation scores mein uncertainty quantify karna

First Principles Se Derivation

Step 1: Sampling Distribution

Maano humara ek population hai jiska true mean $\mu$ (unknown) aur variance ${\sigma }^2$ hai. Hum size $n$ ka sample lete hain, sample mean compute karte hain:

$\bar{X}=\frac{1}{n}{\sum }_{i=1}^n{X}_i$

Yeh step kyun? $\bar{X}$ humara estimator hai $\mu$ ke liye. Lekin $\bar{X}$ khud ek random variable hai kyunki alag-alag samples alag-alag values dete hain.

Central Limit Theorem ke by, bade $n$ ke liye:

$\bar{X}\sim N\left(\mu ,\frac{{\sigma }^2}{n}\right)$

Kyun? Har $X_i$ ka mean $\mu$ aur variance ${\sigma }^2$ hai. Sum ka variance $n{\sigma }^2$ hoga (independent samples). $n$ se divide karne par variance ${\sigma }^2/n$ milta hai. CLT kehta hai distribution normal ho jaati hai.

Step 2: Probability Control Ke Liye Standardize Karo

Hum ek interval $[\bar{X}-\text{margin},\bar{X}+\text{margin}]$ find karna chahte hain jo $\mu$ ko probability $1-\alpha$ ke saath capture kare (jaise 95%).

$\bar{X}$ ko standardize karo:

$Z=\frac{\bar{X}-\mu }{\sigma /\sqrt{n}}\sim N(0,1)$

Yeh step kyun? Standardize karne se humara problem standard normal mein convert ho jaata hai, jahan hum quantiles exactly jaante hain.

$(1-\alpha )$ confidence level ke liye, hum chahte hain:

$P\left(-z_{\alpha /2}\le \frac{\bar{X}-\mu }{\sigma /\sqrt{n}}\le z_{\alpha /2}\right)=1-\alpha$

jahan $z_{\alpha /2}$ critical value hai (jaise 95% ke liye $z_{0.025}=1.96$).

$\alpha /2$ kyun? Hum tail probability ko dono sides par equally split karte hain (two-tailed test).

Step 3: Interval Paane Ke Liye Rearrange Karo

$\sigma /\sqrt{n}$ se multiply karo:

$P\left(-z_{\alpha /2}\cdot \frac{\sigma }{\sqrt{n}}\le \bar{X}-\mu \le z_{\alpha /2}\cdot \frac{\sigma }{\sqrt{n}}\right)=1-\alpha$

Sabse $\bar{X}$ subtract karo:

$P\left(-\bar{X}-z_{\alpha /2}\cdot \frac{\sigma }{\sqrt{n}}\le -\mu \le -\bar{X}+z_{\alpha /2}\cdot \frac{\sigma }{\sqrt{n}}\right)=1-\alpha$

-1 se multiply karo (inequalities flip ho jaati hain):

$P\left(\bar{X}-z_{\alpha /2}\cdot \frac{\sigma }{\sqrt{n}}\le \mu \le \bar{X}+z_{\alpha /2}\cdot \frac{\sigma }{\sqrt{n}}\right)=1-\alpha$

Yeh step kyun? Humne $\mu$ ko beech mein isolate kar diya. Ab interval observed $\bar{x}$ par depend karta hai (lowercase = realized value).

Step 4: Jab σ Unknown Ho (t-Distribution)

Practice mein, hum $\sigma$ nahi jaante. Ise sample standard deviation $s$ se replace karo:

$s=\sqrt{\frac{1}{n-1}{\sum }_{i=1}^n(X_i-\bar{X}{)}^2}$

$n-1$ kyun? Bessel's correction $s^2$ ko ${\sigma }^2$ ka unbiased estimator banata hai (humne $\mu$ estimate karte waqt ek degree of freedom "use kar li").

Ab standardized statistic ek t-distribution follow karta hai $df=n-1$ degrees of freedom ke saath:

$T=\frac{\bar{X}-\mu }{s/\sqrt{n}}\sim t_{n-1}$

Normal kyun nahi? Kyunki $s$ bhi random hai. t-distribution ke heavier tails is extra uncertainty ko account karte hain. Jaise $n\to \infty$, $t_{n-1}\to N(0,1)$.

Worked Examples

Common Mistakes (Apni Galtiyon Ko Samjho)

Practice Mein Isko Kaise Use Karein

Sample Size Planning

Required $n$ find karne ke liye margin of error formula rearrange karo:

$E=z_{\alpha /2}\cdot \frac{\sigma }{\sqrt{n}}⟹n={\left(\frac{z_{\alpha /2}\cdot \sigma }{E}\right)}^2$

Yeh kyun matter karta hai: ±2% margin of error paane ke liye 95% confidence ke saath, maano $\sigma =0.1$:

$n={\left(\frac{1.96\cdot 0.1}{0.02}\right)}^2=96.04\approx 97$

80/20 insight: Precision double karne (yani $E$ half karne) ke liye 4× sample size chahiye kyunki square ki wajah se.

One-sided vs Two-sided

• Two-sided: $\mu$ zyada ya kam ho sakta hai (sabse common)
• One-sided: sirf care karte hain agar $\mu >$ koi value (jaise "kya accuracy 80% se upar hai?")

One-sided 95% CI ke liye: $z_{0.025}=1.96$ ki jagah $z_{0.05}=1.645$ use karo.

Connections Central Limit Theorem – kyun $\bar{X}$ approximately normal hota hai

• Standard Error – kyun hum $\sqrt{n}$ se divide karte hain
• Hypothesis Testing – CIs aur p-values ek hi sikke ke do pehlu hain
• Bootstrap Methods – jab assumptions fail hon tab non-parametric alternative
• Bayesian Credible Intervals – "95% chance" wali interpretation jo tum use karna chahte the
• A/B Testing – treatment effects ke liye confidence intervals
• Cross-validation – model performance estimates mein uncertainty

Active Recall Flashcards

#flashcards/ai-ml

Concept Map