Confidence intervals — derivation for mean, proportion

1. The raw material: the sampling distribution

WHY does variance shrink by $n$ ? Variance of a sum of independent variables adds: $\operatorname{Var}\!\left(\sum X_i\right)=n\sigma^2 \implies \operatorname{Var}\!\left(\tfrac1n\sum X_i\right)=\tfrac{1}{n^2}\cdot n\sigma^2=\frac{\sigma^2}{n}.$ The standard deviation of $\bar X$ — called the standard error — is $\sigma/\sqrt n$ .

HOW do we know its shape? The Central Limit Theorem says for large $n$ , $\bar X \approx \mathcal N\!\left(\mu,\ \frac{\sigma^2}{n}\right) \quad\Longleftrightarrow\quad Z=\frac{\bar X-\mu}{\sigma/\sqrt n}\approx \mathcal N(0,1).$

2. Deriving the CI for the mean (σ known)

Formula Derivation from scratch

Start from the standardized statistic, which is approximately standard normal: $Z=\frac{\bar X-\mu}{\sigma/\sqrt n}.$ Pick a confidence level $1-\alpha$ . By symmetry of the normal, there is a number $z_{\alpha/2}$ with $P\!\left(-z_{\alpha/2}\le Z \le z_{\alpha/2}\right)=1-\alpha.$ Why split $\alpha$ in two? We leave probability $\alpha/2$ in each tail so the central band holds $1-\alpha$ . Substitute $Z$ and algebraically isolate $\mu$ : $-z_{\alpha/2}\le \frac{\bar X-\mu}{\sigma/\sqrt n}\le z_{\alpha/2}$ $-z_{\alpha/2}\frac{\sigma}{\sqrt n}\le \bar X-\mu\le z_{\alpha/2}\frac{\sigma}{\sqrt n}$ $\boxed{\ \bar X - z_{\alpha/2}\frac{\sigma}{\sqrt n}\ \le\ \mu\ \le\ \bar X + z_{\alpha/2}\frac{\sigma}{\sqrt n}\ }$ The half-width $E=z_{\alpha/2}\,\dfrac{\sigma}{\sqrt n}$ is the margin of error.

Common $z_{\alpha/2}$ : 90% → 1.645, 95% → 1.96, 99% → 2.576.

Figure — Confidence intervals — derivation for mean, proportion

3. σ unknown → the t-distribution

WHY swap to $t$ ? When $\sigma$ is unknown we plug in the sample SD $s$ . But $s$ is itself random and tends to underestimate $\sigma$ for small $n$ , so the statistic has fatter tails. The exact distribution (for normal data) is Student's $t$ with $n-1$ degrees of freedom.

As $n\to\infty$ , $t_{n-1}\to \mathcal N(0,1)$ , so for big samples the two methods agree.

4. Deriving the CI for a proportion

WHY is this just a special case? A 0/1 (Bernoulli) variable has mean $p$ and variance $p(1-p)$ . So $\hat p$ is a sample mean of Bernoulli's: $E[\hat p]=p,\qquad \operatorname{Var}(\hat p)=\frac{p(1-p)}{n}.$ By CLT, $\hat p\approx \mathcal N\!\big(p,\ \tfrac{p(1-p)}{n}\big)$ for large $n$ .

5. Worked examples

6. Common mistakes (Steel-man + fix)

7. Active recall

Recall Quick self-test (answer before peeking)

What does "95% confidence" actually mean?
Why does the interval use $\sigma/\sqrt n$ not $\sigma$ ?
When do we use $t$ instead of $z$ ?
What's the variance of a Bernoulli, and how does it give the proportion SE?

Answers: (1) 95% of intervals built this way capture $\mu$ over repeated sampling. (2) We bound the mean, whose SE is $\sigma/\sqrt n$ . (3) When $\sigma$ unknown (esp. small $n$ ). (4) $p(1-p)$ ; SE $=\sqrt{p(1-p)/n}$ , estimated by $\hat p$ .

Recall Feynman: explain to a 12-year-old

Imagine you taste a few spoons of a giant pot of soup to guess its average saltiness. You won't be exactly right, so instead of saying "the soup is exactly 5 salty," you say "I'm pretty sure it's between 4.5 and 5.5." If lots of friends each taste their own spoons and each draw such a range, about 95 out of 100 friends' ranges would actually contain the true saltiness. That range is the confidence interval, and it gets tighter the more spoons you taste (bigger $n$ ).

8. Connections

Central Limit Theorem — engine that makes $\bar X$ normal.
Standard Error — the $\sigma/\sqrt n$ shrinking spread.
Student's t-distribution — fatter tails for estimated $\sigma$ .
Hypothesis Testing — CI ↔ test duality ( $\mu_0$ outside CI ⇔ reject at level $\alpha$ ).
Bernoulli & Binomial Distributions — basis of proportion variance.
Sample Size Determination — invert margin formula to solve for $n$ .

What does a 95% confidence interval actually mean?

Over many repeated samples, 95% of the intervals constructed this way contain the true parameter (a property of the procedure, not of one interval).

Standard error of the sample mean?

\sigma/\sqrt n

(or

s/\sqrt n

when σ unknown).

Why does

\operatorname{Var}(\bar X)=\sigma^2/n

Variance of independent sum adds to

n\sigma^2

; dividing by

n

scales variance by

1/n^2

, giving

\sigma^2/n

CI for mean with σ known?

\bar X \pm z_{\alpha/2}\,\sigma/\sqrt n

CI for mean with σ unknown?

\bar X \pm t_{\alpha/2,n-1}\,s/\sqrt n

Why use

t

instead of

z

s

estimates

\sigma

and adds uncertainty;

t_{n-1}

has fatter tails, becoming normal as

n\to\infty

Why

n-1

degrees of freedom?

One constraint

\sum(X_i-\bar X)=0

is used computing

s

, leaving

n-1

free deviations.

Variance of a Bernoulli(p)?

p(1-p)

CI for a proportion (Wald)?

\hat p \pm z_{\alpha/2}\sqrt{\hat p(1-\hat p)/n}

Validity rule for proportion CI?

n\hat p\ge 5

and

n(1-\hat p)\ge 5

Why split α into α/2 per tail?

Two-sided interval leaves equal probability in each tail so the central band holds exactly

1-\alpha

z_{\alpha/2}

for 95%?

1.96 (NOT 1.645, which is one-sided).

How does margin of error change with n?

Shrinks like

1/\sqrt n

— quadruple n to halve the margin.

Concept Map

Hinglish (regional understanding)

Intuition Hinglish mein samjho

Dekho, confidence interval ka idea simple hai. Hum poori population ka mean $\mu$ nahi jaante, isliye ek sample lekar uska average $\bar X$ nikaalte hain. Lekin $\bar X$ exactly $\mu$ ke barabar kabhi nahi hota — thoda idhar-udhar hota hai. Is "idhar-udhar" ko measure karta hai standard error $=\sigma/\sqrt n$ . Jitna bada sample, utna chhota error, kyunki $\sqrt n$ neeche hai. Isliye CI ki width bhi tab kam hoti hai.

Derivation yaad rakhne ka tarika: $Z=(\bar X-\mu)/(\sigma/\sqrt n)$ approximately standard normal hota hai (CLT ki wajah se). Ab 95% probability ke liye hum bolte hain $-1.96\le Z\le 1.96$ , phir is inequality ko $\mu$ ke liye solve kar dete hain — bas mil gaya formula $\bar X \pm 1.96\,\sigma/\sqrt n$ . Agar $\sigma$ pata nahi hai to uski jagah sample SD $s$ daalo aur $z$ ki jagah $t_{n-1}$ use karo, kyunki tab thodi extra uncertainty hoti hai (isliye $t$ ki tails moti hoti hain).

Proportion ke liye bhi same logic hai — kyunki 0/1 wala Bernoulli variable ka variance $p(1-p)$ hota hai, to $\hat p$ ka SE $=\sqrt{\hat p(1-\hat p)/n}$ ban jaata hai, aur CI $=\hat p \pm 1.96\sqrt{\hat p(1-\hat p)/n}$ .

Sabse important baat (exam aur samajh dono ke liye): "95% confidence" ka matlab yeh nahi hai ki is particular interval mein $\mu$ hone ki 95% chance hai. Matlab yeh hai ki agar tum baar-baar experiment karo, to 100 mein se lagbhag 95 intervals sach mein $\mu$ ko pakad lenge. Yeh procedure ke baare mein statement hai, ek number ke baare mein nahi.