Hypothesis testing and p-values: Visual walkthrough

1.3.20 · D2 · AI-ML › Probability & Statistics › Hypothesis testing and p-values

Yeh page parent topic ka visual companion hai. Hum assume karte hain aapne pehle kabhi bell curve, $Z$ -score, ya "tail" shabd nahi suna. Hum har cheez step by step earn karenge.

Step 1 — Har woh result draw karo jo ek fair coin de sakta hai

KYA. Apna data dekhne se pehle, hum poochhte hain: agar coin sach mein fair hota, toh 100 flips mein kaun se results possible bhi hain? $X$ ko heads ki sankhya maano un 100 flips mein. $X$ ho sakta hai $0, 1, 2, \dots$ , $100$ tak. Kuch counts common hain (around 50), kuch almost impossible hain (jaise 0 ya 100).

KYUN. Hypothesis testing is tarah kaam karta hai ki hum "boring" story ko true maan lete hain — coin fair hai — aur poochhte hain ki hamaara data us story ke against kaisa dikhta hai. Toh pehli cheez jo chahiye woh hai fairness ke under $X$ ki possible values ka poora landscape. Tabhi hum keh sakte hain ki 65 normal hai ya bizarre. Yeh landscape ek probability distribution hai.

PICTURE. Neeche har bar ek possible head-count hai. Bar ki height woh probability hai ki $X$ exactly utne heads ke barabar hoga agar coin fair ho.

Step 2 — Hill almost ek bell curve hai

KYA. Woh bars ka staircase, bade $n$ ke liye, ek smooth twin rakhta hai: ek bell curve (normal distribution). Hum jagged bars ko smooth bell se replace kar lete hain.

KYUN. Do reasons hain. (1) Bell curve ka ek clean formula hai, toh hum 100 binomial bars add karne ki jagah ek table se areas compute kar sakte hain. (2) Central Limit Theorem guarantee karta hai ki yeh swap accurate hai: kyunki $X$ 100 tiny independent $0$ -or- $1$ flips ka sum hai, total hamesha ek bell jaisa dikhta hai. Isliye bell curve yahan sahi tool hai, koi arbitrary curve nahi.

PICTURE. Smooth mint bell almost exactly lavender bars ke upar lie karta hai.

Formula Bell curve ka actual formula (normal PDF)

Smooth mint curve sirf "koi hill" nahi hai — iska ek exact equation hai. Ek value $x$ (ek head-count) par iska height hai $f (x) = fixes total area to 1 \frac{1}{σ 2 π} drops off as you leave the peak e^{- \frac{1}{2} (\frac{x - μ}{σ})^{2}}$

$μ$ woh jagah hai jahan peak baithti hai, $σ$ batata hai kitni wide hai (dono neeche pin down hain).
$\frac{x - μ}{σ}$ peak se widths mein measured distance hai — exactly woh quantity jise hum Step 4 mein $Z$ naam denge. Jab woh $0$ ho (peak par) toh exponent $0$ hai aur $f$ sabse bada hai; jitna door jaate hain, exponent utna negative hota hai, toh curve zero ki taraf jaata hai — yahi bell shape hai.
$e$ Euler's number hai ( $\approx 2.718$ ); ise negative square par raise karna ek smooth symmetric hump banata hai.
Front factor $\frac{1}{σ 2 π}$ sirf woh scaling hai jo curve ke neeche poora area exactly 1 banata hai (total probability 1 honi chahiye). Wider bell → shorter, taki area 1 rahe.

Yahi mint curve hai jiske neeche Step 5 se aage ke saare shaded areas measure hote hain.

Common mistake Swap ek

approximation hai — aur yahan correction in action hai Bell continuous hai (smooth) jabki $X$ discrete hai (sirf whole numbers), toh dono kabhi perfectly match nahi karte — khaaskar tails mein bahut door, jahan ek tiny curve-vs-bars mismatch estimated area ko thoda shift kar sakta hai. Careful kaam mein continuity correction use hoti hai: " $X = 65$ " ka bar $64.5$ se $65.5$ tak span karta hai, toh " $X \geq 65$ " (bars) ko "area beyond $64.5$ " (curve) se best match kiya jaata hai: $P (X \geq 65) \approx P (Z \geq \frac{64.5 - 50}{5}) = P (Z \geq 2.9)$ versus uncorrected $P (Z \geq 3)$ . Dono $\approx 0.0019$ vs $\approx 0.00135$ per tail dete hain — dono threshold se kaafi neeche, toh verdict identical hai. Hum neeche clean $Z = 3$ numbers rakhte hain, lekin ab aapne dekha ki half-unit shift bar-vs-curve mismatch kaise handle karta hai.

Bell ko pin down karne ke liye sirf do numbers chahiye: peak kahan hai (mean $μ$ ) aur kitni wide hai (standard deviation $σ$ ).

Step 3 — Apna real data picture par rakho

KYA. Hum $X = 65$ par ek vertical line daalpete hain — $X$ ki woh value (observed head count) jo humne actually paayi.

KYUN. Distribution "agar fair hota toh kya ho sakta tha" ki duniya hai. Hamaara data us duniya mein $X$ ki ek single value hai. Weirdness judge karne ke liye hume dekhna hoga ki woh hill ke relative kahan land karta hai.

PICTURE. 65 par coral line bell ki right slope par bahut door baithti hai, bulk ke baad.

Dekh ke hi unusual lagta hai — lekin "looks unusual" koi number nahi hai. Step 4 peak se distance ko ek clean, unit-free score mein badalta hai.

Step 4 — "Widths" mein distance measure karo: Z-score

KYA. Hum measure karte hain $X = 65$ peak 50 se kitna door hai, standard deviations mein counted ( $σ$ ki units mein). Us count ko $Z$ bolte hain.

KYUN. "65 average se 15 heads upar hai" kehna apne aap mein meaningless hai — 15 heads narrow bell ke liye bahut bada hai aur wide bell ke liye tiny. Width $σ$ se divide karne par distance har problem mein comparable ho jaata hai. $Z$ of 3 ka matlab hamesha "3 widths out" hai, chahe hum coins flip karein, heights measure karein, ya models test karein. Yeh exactly wahi standardizing move hai jo confidence intervals mein use hoti hai.

PICTURE. Horizontal axis ab widths mein relabelled hai: peak par 0, aur 65 $Z = 3$ par baitha hai.

Step 5 — "Utna extreme ya zyada": kyun hum ek tail shade karte hain

KYA. P-value exactly 65 paane ki probability nahi hai. Yeh kam se kam 65 jaisi weird result paane ki probability hai. Toh hum poori tail shade karte hain — har woh outcome $Z \geq 3$ — sirf single line nahi.

KYUN. Koi bhi exact outcome (chahe sabse likely wala bhi) ki probability chhoti hoti hai, toh "exactly 65 ki probability" ek bura surprise-meter hoga. Jo hum truly jaanna chahte hain woh hai: kitni baar ek fair coin humein itna ya zyada surprise karega? Yeh $N (0, 1)$ tail ke neeche ek area hai, hamaari line se edge tak baahir.

PICTURE. $Z = 3$ se rightward sab kuch coral shade hai — "at least this weird" region.

Standard normal ke neeche us shaded right tail ka area ek known number hai: $P (Z \geq 3) \approx 0.00135$

Step 6 — Dono directions: two-tailed p-value

KYA. Hamaara alternative $H_{1} : p \neq = 0.5$ tha — coin kisi bhi taraf biased ho sakta hai. Toh "65 heads jaisa weird" uske mirror image ko bhi include karta hai: 35 heads ( $Z = - 3$ ), tails ki taraf equally strong bias. Hum dono tails shade karte hain aur unke areas add karte hain.

KYUN. Humne ek two-sided sawaal poochha tha ("kya yeh kisi bhi taraf biased hai?"), toh ek result jo tails direction mein equally extreme hai woh equally surprising count hona chahiye. Left tail ignore karna secretly hamare sawaal ko ek one-sided mein badal deta. Kyunki standard normal symmetric hai, left tail ka area right ke barabar hai, toh hum simply double kar dete hain.

PICTURE. Do symmetric coral tails, ek $Z = + 3$ par aur uska twin $Z = - 3$ par.

Words mein padhna: agar coin perfectly fair hota, toh hum itna extreme result sirf 0.27 % baar dekhte. Yeh rare hai.

Step 7 — Verdict: threshold $α$ se compare karo

KYA. Hum pehle se ek line-in-the-sand choose karte hain — significance level $α$ , usually $0.05$ — aur fair-coin story tab hi reject karte hain jab hamaara p-value us se neeche gire.

KYUN. Humein ek rule chahiye jo pehle se agree ho taki data dekhne ke baad goalposts move na kar sakein. $α = 0.05$ ka matlab hai "main 5 % time luck se bewaqoof banne ko accept karunga." $α$ choose karna actually choose karna hai ki aap kitni baar false alarm tolerate karte ho — ek Type I error.

PICTURE. Ek number line: tiny p-value $0.0027$ $α = 0.05$ gate se kaafi left mein baitha hai. Gate ke andar → reject.

$p = 0.0027 < 0.05 = α ⟹ reject H_{0}$

Conclusion: strong evidence ki coin fair nahi hai.

Step 8 — Edge aur degenerate cases (picture extremes par kaisi dikhti hai)

KYA. Hum corners check karte hain taki koi scenario surprise na kare.

KYUN. Ek derivation jis par aap trust karte ho woh apni khud ki limits mein survive karni chahiye.

PICTURE. Chaar mini-panels: data peak par, data mild bulge par, one-tailed shade, aur $p$ ka "wall" jahan woh 0 tak kabhi nahi pohunchta.

Data exactly peak par ( $X = 50$ , $Z = 0$ ). Dono tails poori bell hain, toh $p = 2 \cdot 0.5 = 1$ . Sabse bada possible p-value $1$ hai — perfectly ordinary data "weird" kabhi nahi ho sakta, toh hum kabhi reject nahi karte. ✔
Mildly off-centre ( $X = 54$ , $Z = 0.8$ ). Tails moti hain, $p \approx 0.42 > 0.05$ → fail to reject. 50 se thoda upar hona luck se easily explain ho jaata hai.
One-tailed sawaal. Agar $H_{1}$ hoti "sirf heads ki taraf biased" toh hum sirf right tail shade karte: $p = P (Z \geq 3) \approx 0.00135$ , two-tailed value ka aadha. Same data, alag sawaal, alag p-value — toh hypothesis hamesha compute karne se pehle fix karo.
p kabhi 0 nahi ho sakta. Data kitna bhi extreme ho, standard-normal tail ka kuch area hota hai — coral kabhi poori tarah vanish nahi hoti. Toh hum fairness reject kar sakte hain lekin use poori certainty ke saath disprove nahi kar sakte.

One-picture summary

Har step, compressed: fair-coin bell (Step 2), $Z = 3$ par hamaari data line (Step 4), dono shaded tails (Step 6), aur verdict $p = 0.0027 < 0.05$ (Step 7) — sab ek canvas par.

Recall Feynman retelling — kisi dost ko batao

Ek fair coin, 100 baar flip kiya, usually 50 heads ke paas land karta hai — exact chances binomial formula $(k 100) 0. 5^{100}$ se aati hain, aur main inhe 50 par centred ek bell-shaped hill ki tarah draw kar sakta hoon, kareeb 5 heads wide. Maine apna flip kiya aur $X = 65$ mila. Meri hill par, 65 teen "widths" right mein door hai — main ise ek line se mark karta hoon, aur 0 par re-centre karke aur width ko 1 par rescale karke, woh spot standard bell (mean 0, width 1) par $Z = 3$ kehlata hai. Ab key move: main nahi poochhta "exactly 65 ka chance kya hai?" (silly — har exact number rare hai). Main poochhta hoon "kitna chance hai ki ek fair coin mujhe itna ya zyada surprise kare?" — yeh far-right tail mein shaded area hai, plus left tail mein uska mirror kyunki tails ki taraf bias mujhe utna hi surprise karega. Woh dono slivers milke all fair-coin worlds ka kareeb 0.27 % cover karte hain. Kyunki maine pehle se agree kiya tha ki sirf tab "unfair!" boloon jab surprise 5 % se neeche ho, aur 0.27 % kaafi neeche hai, main fair-coin story reject karta hoon. Maine bias prove nahi kiya — tail kabhi zero tak nahi pohunchti, toh certainty table se bahar hai — lekin fairness ab jo maine dekha uski ek bekar explanation hai.

Recall Quick self-test

Step 1 mein ek bar ki height kya represent karti hai, aur kaunsa formula deta hai? ::: Probability $P (X = k) = (k n) p^{k} (1 - p)^{n - k}$ ki head count $X$ exactly $k$ ke barabar hoga agar coin fair ho. Smooth bell curve ka formula kya hai, aur uska front factor kya karta hai? ::: $f (x) = \frac{1}{σ 2 π} e^{- \frac{1}{2} ((x - μ) / σ)^{2}}$ ; front factor $\frac{1}{σ 2 π}$ curve ko scale karta hai taki uska total area 1 ho. $Z$ get karne ke liye $σ$ se kyun divide karte hain? ::: Peak se distance widths mein measure karne ke liye, jisse yeh har problem mein comparable ho jaata hai — aur curve standard normal $N (0, 1)$ mein turn ho jaati hai. $H_{0}$ ke under $Z$ kaun si distribution follow karta hai? ::: Standard normal distribution, mean $0$ aur standard deviation $1$ , $Z \sim N (0, 1)$ . Hum poori tail shade kyun karte hain, na ki ek single line? ::: P-value data ke kam se kam utne extreme hone ka chance hai, jo ek area hai, na ki ek single-point probability. Two-tailed p-value mein 2 ka factor kyun? ::: Kyunki $H_{1} : p \neq = 0.5$ dono directions mein extremes count karta hai; symmetric bell dono tails ko equal banati hai, toh hum ek ko double karte hain. Normal approximation kab safe hai? ::: Jab both $n p \geq 10$ aur $n (1 - p) \geq 10$ ho; warna exact binomial use karo. P-value sabse zyada kitna ho sakta hai, aur kab? ::: $1$ — jab data exactly peak par ho ( $Z = 0$ ), sabse ordinary possible result.

Related building blocks: 1.3.1-Random-variables-and-distributions · 1.3.15-Central-limit-theorem · 1.3.18-Confidence-intervals · 1.3.21-Type-I-and-Type-II-errors · 3.2.12-Multiple-testing-correction · 2.5.7-Statistical-significance-in-experiments