Hypothesis testing — null - alternative, test statistic, p-value, errors (Type I & II)

4.9.20 · Maths › Probability Theory & Statistics

Hypothesis testing exist kyun karta hai?

Hamare paas ek sample hai, lekin hum poori population ke baare mein ek decision lena chahte hain. Sample noisy hota hai: chahe kuch bhi special na ho raha ho, random fluctuation patterns produce kar deta hai. Isliye humein ek disciplined rule chahiye jo control kare ki hum kitni baar randomness se fool ho jaate hain. Hypothesis testing exactly wahi rule hai.

Test statistic kya hota hai? Ek akela number jo data se compute hota hai aur measure karta hai ki sample $H_{0}$ ki prediction se kitna door hai, standard error ke units mein measured taaki hum iske random distribution ko jaanein.

z-test statistic ko first principles se derive kaise karein

Hum $H_{0} : μ = μ_{0}$ test karna chahte hain, sample mean $\overset{ˉ}{X}$ use karke, $n$ observations se, aur known population SD $σ$ ke saath.

Step 1 — $\overset{ˉ}{X}$ ki distribution. Har $X_{i}$ ka mean $μ$ aur variance $σ^{2}$ hai. Sample mean hai $\overset{ˉ}{X} = \frac{1}{n} \sum X_{i}$ .

$E [\overset{ˉ}{X}] = μ, Var (\overset{ˉ}{X}) = \frac{1}{n ^{2}} \sum Var (X_{i}) = \frac{σ ^{2}}{n} .$ Yeh step kyun? Independent variables ke sum ka variance add hota hai; $1/ n^{2}$ constant factor ke square se aata hai.

Step 2 — standard error. Toh $\overset{ˉ}{X}$ ka spread hai $SE = σ / n$ . Kyun? SD variance ka square root hota hai.

Step 3 — $H_{0}$ ke under standardize karein. Maano $H_{0}$ sach hai, toh $μ = μ_{0}$ . Central Limit Theorem ke wajah se, $\overset{ˉ}{X}$ approximately normal hai, isliye

$Z = \frac{X ˉ - μ _{0}}{σ / n} \sim N (0, 1) .$ Yeh step kyun? Assumed mean subtract karo, SE se divide karo → ek unit-free score jo bataata hai ki data $H_{0}$ se kitne standard errors door hai.

p-value: itna badly fool hone ki probability

Figure — Hypothesis testing — null - alternative, test statistic, p-value, errors (Type I & II)

Galat hone ke do tarike

Definition Type I & Type II errors

	$H_{0}$ true	$H_{0}$ false
Reject $H_{0}$	❌ Type I error ( $α$ )	✅ correct (power $1 - β$ )
Fail to reject	✅ correct ( $1 - α$ )	❌ Type II error ( $β$ )

Type I ( $α$ ) = false alarm = ek innocent insaan ko convict karna.
Type II ( $β$ ) = miss = ek guilty insaan ko free jaane dena.
Power $= 1 - β$ = ek real effect ko correctly detect karne ki probability.

Tradeoff kyun hoti hai: $α$ kam karna (reject karne mein stricter hona) critical cutoff ko aur door push kar deta hai, jisse true effects detect karna mushkil ho jaata hai → $β$ badhta hai. Dono ko shrink karne ka ek hi tarika hai — $n$ badhao (dono distributions ko narrow karta hai).

Worked Example 1 — z-test, two-tailed

Ek machine ko bottles $μ_{0} = 500$ ml mein fill karni chahiye, $σ = 8$ ml. Hum $n = 64$ bottles sample karte hain, $\overset{ˉ}{X} = 497$ ml milta hai. $α = 0.05$ pe test karo ki machine off hai ya nahi.

$H_{0} : μ = 500$ , $H_{1} : μ \neq = 500$ . Two-tailed kyun? "Off" ka matlab hai ya too high ya too low.
$SE = 8/ 64 = 1$ . Kyun? $σ / n$ .
$z = \frac{497 - 500}{1} = - 3$ . Kyun? Observed mean ko standardize karo.
$p = 2 P (Z \geq 3) \approx 2 (0.00135) = 0.0027$ . Double kyun? Dono tails "utni extreme" hain.
$0.0027 < 0.05$ → $H_{0}$ reject. Machine mis-calibrated hai.

Worked Example 2 — right-tailed t-test

Ek nayi teaching method claim karti hai ki scores purane mean $μ_{0} = 70$ se zyada honge. Sample: $n = 16$ , $\overset{ˉ}{X} = 74$ , $s = 6$ . $α = 0.05$ pe test karo.

$H_{0} : μ = 70$ , $H_{1} : μ > 70$ . Right-tailed kyun? Claim sirf "higher" ki hai.
$T = \frac{74 - 70}{6/ 16} = \frac{4}{1.5} = 2.667$ with $df = 15$ . $t$ kyun, $z$ nahi? $σ$ unknown hai.
Critical $t_{0.05, 15} \approx 1.753$ . Kyunki $2.667 > 1.753$ , $p < 0.05$ → $H_{0}$ reject: evidence hai ki method help karta hai.

Worked Example 3 — non-rejection interpret karna

Ex.2 jaisa hi lekin $\overset{ˉ}{X} = 71$ : $T = \frac{1}{1.5} = 0.667 < 1.753$ → fail to reject. Yeh wording kyun? Humne method ko useless prove nahi kiya; bas evidence nahi mila. Evidence ka absence ≠ absence ka evidence.

Common mistake Classic blunders ko steel-man karna

"p-value = $H_{0}$ ke sach hone ki probability." Sahi kyun lagta hai: chhota $p$ aisa lagta hai jaise " $H_{0}$ probably false hai," toh $p$ $H_{0}$ ki probability measure karta lagta hai. Fix: $p$ $H_{0}$ ko sach maanke compute hota hai — yeh $P (data ∣ H_{0})$ hai, $P (H_{0} ∣ data)$ nahi. Inhe flip karna prosecutor's fallacy hai.

"Fail to reject = $H_{0}$ accept karna / prove karna." Sahi kyun lagta hai: agar reject nahi kiya, toh surely endorse kar rahe ho. Fix: test mein sirf effects detect karne ki power thi; null result ka matlab sirf chhota $n$ ho sakta hai. Hum kehte hain "insufficient evidence."

" $α = 0.05$ matlab meri 5% rejections galat hain." Sahi kyun lagta hai: dono mein 5% aur errors involved hain. Fix: $α$ hai $P (reject ∣ H_{0} true)$ — $H_{0}$ ke sach hone ke conditional, saari rejections ka share nahi.

Data dekhne ke baad tail choose karna. One-tailed isliye choose karna ki significance milti hai — yeh cheating hai; hypothesis direction dekhne se pehle set honi chahiye.

Recall Feynman: ek 12-saal ke bachche ko samjhao

Socho tumhe shak hai ki ek coin rigged hai. Tum maanke shuru karte ho ki yeh fair hai (woh null hai). Tum ise 100 baar flip karte ho aur 80 heads milte hain. Ab poochho: "Agar coin sach mein fair hota, toh pure luck se kitni baar aisa ek-taraf jhukaa result milta?" Agar jawaab hai "almost never" (tiny p-value), toh tum believe karna band kar dete ho ki yeh fair hai. Agar "eh, kabhi kabhi hota hai," toh tum shrug karte ho aur fair maante rehte ho — prove isliye nahi kiya, balki doubt karne ki koi achi wajah nahi mili. Type I error hai ek perfectly fair coin ke baare mein "rigged!" chillana. Type II error hai ek loaded coin ko fair kehna.

Active Recall

Equality kismein hoti hai,

H_{0}

mein ya

H_{1}

mein?

Null hypothesis

H_{0}

mein.

p-value precisely define karo.

P (test statistic as or more extreme than observed ∣ H_{0} true)

Significance level

α

ke saath decision rule?

H_{0}

reject karo agar

p \leq α

; warna fail to reject.

μ = μ_{0}

test karne ke liye z-statistic derive karo.

\overset{ˉ}{X}

ka mean

μ_{0}

hai, variance

σ^{2} / n

H_{0}

ke under; standardize karo:

Z = (\overset{ˉ}{X} - μ_{0}) / (σ / n) \sim N (0, 1)

z

ki jagah

t

kab use karte hain?

Jab population

σ

unknown ho aur sample

s

se estimate karna pade; tab

T \sim t_{n - 1}

Type I error kya hai aur uski probability kya hai?

Sach

H_{0}

ko reject karna (false alarm); probability

α

Type II error kya hai aur uski probability kya hai?

False

H_{0}

ko reject karne mein fail hona (miss); probability

β

Statistical power define karo.

1 - β

= false

H_{0}

ko correctly reject karne ki probability.

Hum

α

aur

β

dono freely kyun nahi ghata sakte?

Stricter cutoff

α

kam karta hai lekin

β

badhata hai; sirf bada

n

dono ko shrink karta hai.

z_{o b s}

se two-tailed p-value?

p = 2 P (Z \geq ∣ z_{o b s} ∣)

"Fail to reject" ko "accept" kyun nahi kehte?

Test mein power nahi hoti; evidence ka absence, absence ka evidence nahi hai.

Yahan prosecutor's fallacy kya hai?

P (data ∣ H_{0})

(the p-value) ko

P (H_{0} ∣ data)

se confuse karna.

Connections

Central Limit Theorem — $\overset{ˉ}{X}$ ki normal distribution justify karta hai.
Standard Error — har test statistic ka denominator.
Confidence Intervals — two-tailed tests ka dual (reject ⇔ $μ_{0}$ CI ke bahar).
Normal Distribution aur Student t-distribution — reference distributions.
Bayesian Inference — $P (H_{0} ∣ data)$ directly deta hai (jo p-values nahi dete).
Statistical Power & Sample Size — $β$ control karna.