Hypothesis testing and p-values: Worked examples — Hinglish deep dive

1.3.20 · D3 · AI-ML › Probability & Statistics › Hypothesis testing aur p-values

Yeh page ek worked-example gauntlet hai. Parent note ne tumhe machinery ek baar dikhaayi. Yahan hum use har us shape ke through run karte hain jo ek problem le sakti hai: one tail, two tails, woh boundary jahan p-value exactly $α$ ke barabar hoti hai, ek degenerate "zero difference" input, ek limiting "huge sample" case, ek real-world word problem, aur ek exam-style trap. Agar tum yahan koi case dekh lete ho, toh tumne use pehle se hi dekh liya hai.

Shuru karne se pehle, ek promise: neeche use kiya gaya har symbol ya toh parent mein define hai ya yahan re-earn kiya gaya hai. Jab bhi koi number nikalti hai, use verify block mein check kiya jaata hai.

The scenario matrix

Hypothesis test ko ek machine ki tarah socho jisme knobs hain. Knobs hain: hum kaun sa tail(s) dekhte hain, observed effect ka sign, aur input kitna extreme hai. Neeche ki table mein har knob-setting list ki gayi hai taaki baad mein kuch bhi surprise na kare.

Cell	Case class	Kyun tricky hai	Covered by
A	Two-tailed, positive effect	tail double karni padti hai; direction matter nahi karta	Ex 1
B	Two-tailed, negative effect	$z$ negative hai — kya hum sign flip karte hain?	Ex 2
C	One-tailed, right side	sirf "bigger" count hota hai; double mat karo	Ex 3
D	One-tailed, wrong direction	effect us taraf jaata hai jo tumne umeed ki thi uske opposite	Ex 4
E	Boundary case: p-value $= α$	reject karein ya nahi? " $<$ vs $\leq$ " ka edge	Ex 5
F	Zero / degenerate input	observed = null exactly, $z = 0$	Ex 6
G	Limiting case: huge $n$	tiny effect "significant" ban jaata hai	Ex 7
H	Real-world word problem	tumhe khud $H_{0}, H_{1}$ banane padte hain	Ex 8
I	Exam twist: given a target p-value, find the cutoff	machine ko ulta chalao	Ex 9

Teen definitions jinpar hum poori tarah rely karte hain, plain words mein re-stated:

Definition Vocabulary, re-earned

Null hypothesis $H_{0}$ : woh boring "kuch nahi chal raha" claim jo hum compute karte waqt true maante hain.
$π$ (true proportion), $π_{0}$ (uski null value), $\overset{π}{^}$ (sample estimate): $π$ woh real chance hai jo hum dekh nahi sakte; $π_{0}$ woh number hai jo $H_{0}$ claim karta hai ki yeh equal hai; $\overset{π}{^}$ woh fraction hai jo humne actually observe kiya.
$X$ (count of successes): "yes" outcomes ki raw number jo humne count ki (jaise number of heads). Yeh $0$ aur $n$ ke beech ek whole number hai.
$Z$ vs $z$ : capital $Z$ random variable hai — statistics ka woh poora bell-shaped spread jo hum pa sakte agar hum experiment repeat karte. Lowercase $z$ ek fixed observed number hai — woh value jo $Z$ ne hamare actual data ke liye li. Isliye " $P (Z \geq z)$ " padha jaata hai: "random $Z$ ke hamare ek observed $z$ par ya usse aage land karne ki chance."
$SE$ (standard error): sample estimate ka standard deviation — estimate sample se sample tak kitna wobble karega. Ek proportion ke liye, $SE = π_{0} (1 - π_{0}) / n$ . Yeh woh "width" hai jisse hum standardize karte waqt divide karte hain.
$z$ (the test statistic): "mera observation $H_{0}$ ki prediction se kitne standard errors dur hai?" Surprise measure karne ka ek ruler, $z = (observed - null) / SE$ se compute kiya jaata hai.
p-value: probability, agar $H_{0}$ true hoti, ki ek result kam se kam itna surprising ho. Chota p-value = mera data boring story ke liye achha fit nahi hai.
$α$ (significance level): lakeer jo rekhaa mein khinchi gayi hai. Agar p-value $< α$ toh hum $H_{0}$ reject karte hain. Usually $α = 0.05$ .

Neeche ki picture poora game hai: ek bell curve ( $H_{0}$ ke under random variable $Z$ ki distribution), tumhara ek observed value $z$ uspar mark kiya hua, aur shaded tail area = the p-value.

Figure — Hypothesis testing and p-values

Yahan se sab kuch hai: $z$ dhundo, sahi region shade karo, area padho. Yeh dekhne ke liye ki $Z$ bell-shaped kyun hai, 1.3.15-Central-limit-theorem dekho, aur "random variable" aur "distribution" ka matlab jaanne ke liye 1.3.1-Random-variables-and-distributions dekho.

Cell A — Two-tailed, positive effect

Worked example Ex 1 · Ek coin bahut zyada heads de raha hai

$n = 100$ baar flip karo, aur $X$ ko heads ka count (successes) maano. Hum $X = 64$ observe karte hain. Test karo $H_{0} : π = 0.5$ vs $H_{1} : π \neq = 0.5$ .

Forecast: expected 50 se zyada heads, isliye $z > 0$ . Two-tailed ka matlab hai hum ek tail ko double karte hain. Guess karo: kya p-value $0.05$ se upar hai ya neeche?

$H_{0}$ ke under Mean aur spread. $π_{0} = 0.5$ ke saath: $μ = n π_{0} = 100 (0.5) = 50$ , $σ = n π_{0} (1 - π_{0}) = 25 = 5$ . Yeh step kyun? Yeh bell ka centre aur width hain jo count $X$ follow karti agar coin fair hota — "normal" ke liye yardstick.
Standardize karo. $z = \frac{X - μ}{σ} = \frac{64 - 50}{5} = \frac{14}{5} = 2.8$ . Yeh step kyun? "64 heads" ko "centre se 2.8 std-devs upar" mein convert karta hai, ek scale jise hum look up kar sakte hain.
Dono tails. $p-value = 2 \cdot P (Z \geq 2.8) = 2 (0.002555) = 0.00511$ . Yeh step kyun? $H_{1}$ kehta hai "kisi bhi taraf biased," isliye 36 heads ka ek symmetric result equally weird hoga — hum dono tails count karte hain. " $P (Z \geq 2.8)$ " padho: "random $Z$ ke hamare observed $z = 2.8$ se aage jaane ki chance."
Decide karo. $0.00511 < 0.05 \Rightarrow$ reject $H_{0}$ .

Verify: $z = 2.8$ "2 std-dev" rule-of-thumb (jo p-value $\approx 0.05$ deta hai) se kaafi aage baitha hai, isliye ek chota p-value sanity-consistent hai. Units: $z$ dimensionless hai ✓.

Cell B — Two-tailed, negative effect

Worked example Ex 2 · Ek coin bahut KAM heads de raha hai

Wahi coin, $n = 100$ , heads ka count ab $X = 36$ . Same two-tailed test.

Forecast: kam heads → $z < 0$ . Kya negative sign p-value change karta hai? Padhne se pehle guess karo.

Standardize karo. $z = \frac{X - μ}{σ} = \frac{36 - 50}{5} = \frac{- 14}{5} = - 2.8$ . Yeh step kyun? Same yardstick; sign sirf batata hai ki hum centre ke left par hain.
Two-tailed test ke liye magnitude lo. $p-value = 2 \cdot P (Z \geq ∣ - 2.8 ∣) = 2 \cdot P (Z \geq 2.8) = 0.00511$ . Yeh step kyun? Bell symmetric hai — $- 2.8$ se aage left tail ka exactly wahi area hai jaisa $+ 2.8$ se aage right tail ka. Isliye two-tailed p-value $z$ se nahi, $∣ z ∣$ se drive hoti hai.
Decide karo. Ex 1 jaisa: reject $H_{0}$ .

Verify: Ex 1 aur Ex 2 identical p-value dete hain ( $0.00511$ ). Yahi two-tailed ka poora point hai: direction irrelevant hai, sirf centre se distance matter karta hai ✓.

Cell C — One-tailed, right side

Worked example Ex 3 · Kya naya model actually better hai?

Old model: 82% accuracy. New model: sample accuracy $\overset{π}{^} = 0.87$ on $n = 200$ samples. $H_{0} : π = π_{0} = 0.82$ , $H_{1} : π > 0.82$ (hum sirf care karte agar improve hua). Yahan $π_{0} = 0.82$ null proportion hai — woh accuracy jo $H_{0}$ claim karta hai ki model mein actually hai.

Forecast: improvement positive hai, sirf ek tail, isliye hum double nahi karte. Kya yeh $0.05$ clear karega?

$H_{0}$ ke under Standard error. Vocabulary block se yaad karo ki $SE$ (standard error) estimate ka wobble hai: $SE = \frac{π _{0} ( 1 - π _{0} )}{n} = \frac{0.82 \cdot 0.18}{200} = \frac{0.1476}{200} = 0.000738 = 0.027166$ . Yeh step kyun? Jab hum ek proportion test karte hain, uski apni std-dev $1/ n$ ki tarah shrink hoti hai. Yeh $SE$ possible accuracies ki bell ki width hai.
Standardize karo. $z = \frac{π ^ - π _{0}}{SE} = \frac{0.87 - 0.82}{0.027166} = \frac{0.05}{0.027166} = 1.8405$ . Yeh step kyun? "5 percentage points better" ko $SE$ units mein convert karta hai.
Sirf ek tail. $p-value = P (Z \geq 1.8405) = 0.03285$ . Yeh step kyun? $H_{1}$ one-directional hai — ek worse model kabhi humein reject karwa nahi sakta, isliye uski tail count nahi hoti.
Decide karo. $0.0329 < 0.05 \Rightarrow$ reject $H_{0}$ . Gain significant hai.

Verify: agar humne galti se yeh two-tailed run kiya hota toh hum $2 (0.0329) = 0.066 > 0.05$ paate aur reject karne mein fail ho jaate — isliye sahi tail-count choose karna yahan genuinely decision change kar deta hai ✓.

Cell D — One-tailed, effect GALAT direction mein

Worked example Ex 4 · Naya model actually worse hai

Same one-tailed test $H_{1} : π > 0.82$ with $π_{0} = 0.82$ , lekin naya model $n = 200$ par $\overset{π}{^} = 0.78$ score karta hai. Standard error $SE$ Ex 3 se unchanged hai ( $0.027166$ ) kyunki yeh sirf $π_{0}$ aur $n$ par depend karta hai.

Forecast: model worse ho gaya, lekin hamara test sirf improvement "sunta" hai. P-value ho bhi sakta hai kya? Guess karo ki hum reject kar sakte hain ya nahi.

Standardize karo. $z = \frac{π ^ - π _{0}}{SE} = \frac{0.78 - 0.82}{0.027166} = \frac{- 0.04}{0.027166} = - 1.4724$ . Yeh step kyun? Negative $z$ = observation null se neeche hai — $H_{1}$ jo predict karta hai uske opposite side par.
Right-tail p-value. $p-value = P (Z \geq - 1.4724) = 0.9295$ . Yeh step kyun? Right-tailed test measure karta hai "bell ka kitna hissa meri $z$ se upar hai." Kyunki $z$ left par hai, almost poori bell uske upar hai → huge p-value.
Decide karo. $0.9295 > 0.05 \Rightarrow$ fail to reject $H_{0}$ , correctly. Ek worse model kabhi "better" ka evidence nahi ho sakta.

Verify: $P (Z \geq - 1.4724) = 1 - P (Z \geq 1.4724) = 1 - 0.0705 = 0.9295$ ✓. $0.5$ se upar ka p-value tell-tale sign hai ki tumhara effect one-tailed test ke liye galat way gaya.

Cell E — Exact boundary: p-value $= α$

Worked example Ex 5 · Exactly line par landing karna

Ek test $α = 0.05$ ke saath two-tailed test mein $z = 1.959964$ yield karta hai. Reject karein ya nahi?

Forecast: $z \approx 1.96$ famous "95% critical value" hai. Exactly-on-the-line reject karta hai?

P-value compute karo. $p-value = 2 \cdot P (Z \geq 1.959964) = 2 (0.025) = 0.05$ . Yeh step kyun? $1.959964$ is tarah define kiya gaya hai ki har tail exactly $2.5%$ hold kare; double karne par exactly $5%$ hota hai.
Rule literally apply karo. Decision rule hai "reject if the p-value $< α$ " — strict inequality. Yahan p-value $= 0.05$ nahi hai $0.05$ se kam. Yeh step kyun? Convention $<$ hai, $\leq$ nahi. Exact boundary par hum fail to reject karte hain.
Decide karo. p-value $= 0.05 \neq < 0.05 \Rightarrow$ fail to reject $H_{0}$ (ek baal se).

Verify: woh critical value jo p-value $= α = 0.05$ two-tailed banata hai $z^{*} = 1.959964$ hai; hamara $z$ exactly uske barabar hai, isliye p-value $= α$ exactly hai ✓. Yahi edge hai kyun papers p-value ko kai digits tak report karte hain.

Cell F — Degenerate input: observation null ke barabar hai

Worked example Ex 6 · Tum exactly expected value observe karte ho

$n = 100$ baar flip karo aur exactly $X = 50$ heads ka count lo. Two-tailed $H_{0} : π = 0.5$ .

Forecast: data perfectly boring hai. $z$ kya hai? P-value se zyada se zyada kya ho sakta hai?

Standardize karo. $z = \frac{X - μ}{σ} = \frac{50 - 50}{5} = \frac{0}{5} = 0$ . Yeh step kyun? Mean se zero deviation → zero surprise.
P-value. $p-value = 2 \cdot P (Z \geq 0) = 2 (0.5) = 1.0$ . Yeh step kyun? Bell ka aadha hissa centre ke upar hota hai. "Kam se kam kisi bhi deviation se extreme nahi" practically har possible outcome include karta hai → probability 1.
Decide karo. p-value $= 1.0 > 0.05 \Rightarrow$ fail to reject. Bilkul — data exactly wahi tha jo $H_{0}$ ne predict kiya tha.

Verify: p-value exactly $1$ par max hota hai jab $z = 0$ ; yeh kabhi $1$ se exceed nahi kar sakta (yeh ek probability hai) ✓. Yeh har test ka "sanity floor" hai.

Cell G — Limiting case: huge sample size

Worked example Ex 7 · Ek tiny effect "significant" ban jaata hai

Same 82%→82.5% question, lekin ab $n = 100000$ . Toh $\overset{π}{^} = 0.825$ , $π_{0} = 0.82$ . One-tailed $H_{1} : π > 0.82$ .

Forecast: effect (half a percentage point) trivial hai. Lekin $n$ enormous hai. $z$ ka kya hoga?

Standard error shrink karta hai. $SE = \frac{π _{0} ( 1 - π _{0} )}{n} = \frac{0.82 \cdot 0.18}{100000} = 0.000001476 = 0.0012149$ . Yeh step kyun? Kyunki $SE \propto 1/ n$ , $n$ ko 500 se multiply karne par (200 se) $SE$ $500 \approx 22 \times$ shrink ho jaata hai.
Standardize karo. $z = \frac{π ^ - π _{0}}{SE} = \frac{0.825 - 0.82}{0.0012149} = \frac{0.005}{0.0012149} = 4.1156$ . Yeh step kyun? Same tiny numerator (0.005) bahut chote denominator par → huge $z$ .
P-value. $p-value = P (Z \geq 4.1156) = 0.00001929$ . Yeh step kyun? 4 se aage $z$ tail mein deep hai; area minuscule hai.
Decide karo. p-value $≪ 0.05 \Rightarrow$ reject $H_{0}$ . "Statistically significant"… phir bhi effect sirf $0.5%$ hai.

Verify: Ex 3 se compare karo (same effect size $0.05$ raw terms mein, $n = 200$ , $z = 1.84$ ). Yahan effect $10 \times$ chota hai phir bhi $z$ $2.2 \times$ bada hai, purely $n$ se ✓.

Cell H — Real-world word problem (tum hypotheses banate ho)

Worked example Ex 8 · Kya naya checkout page cart abandonment reduce karta hai?

Historically 30% users apna cart abandon karte hain. Ek redesign ke baad, $n = 400$ sessions mein, $X = 108$ abandon karte hain (yahan $X$ abandonments ka count hai). Kya redesign ne abandonment lower kiya?

Forecast: tumhe English → $H_{0}, H_{1}$ translate karna hoga. "Lower" ek direction hai → one-tailed. Kaun sa tail?

Frame karo. $H_{0} : π = π_{0} = 0.30$ (redesign ne kuch nahi kiya). $H_{1} : π < 0.30$ (redesign ne abandonment reduce kiya). One-tailed, left. Yeh step kyun? Business question sirf improvement (ek drop) ki care karta hai. Hum left tail par sun rahe hain.
Observed rate. $\overset{π}{^} = X / n = 108/400 = 0.27$ . Yeh step kyun? Raw count $X$ ko proportion mein convert karo jo test $π_{0}$ se compare karta hai.
Standard error. $SE = \frac{π _{0} ( 1 - π _{0} )}{n} = \frac{0.30 \cdot 0.70}{400} = \frac{0.21}{400} = 0.000525 = 0.022913$ .
Standardize karo. $z = \frac{π ^ - π _{0}}{SE} = \frac{0.27 - 0.30}{0.022913} = \frac{- 0.03}{0.022913} = - 1.30931$ . Yeh step kyun? Negative $z$ yahan acchi khabar hai — abandonment drop hua, jo woh direction hai jo $H_{1}$ predict karta hai.
Left-tail p-value. $p-value = P (Z \leq - 1.30931) = 0.09524$ . Yeh step kyun? $H_{1} : π < π_{0}$ ke liye hum apne $z$ ke neeche ka area measure karte hain — "aur bhi bada drop" region.
Decide karo. $0.0952 > 0.05 \Rightarrow$ fail to reject $H_{0}$ . Encouraging drop, lekin $α = 0.05$ par abhi convincing nahi.

Verify: $P (Z \leq - 1.30931) = P (Z \geq 1.30931) = 0.09524$ symmetry se ✓. Practical note: yeh "experiment abhi mat rokna, aur data collect karo" result hai, "redesign fail ho gaya" nahi.

Cell I — Exam twist: machine ko ulta chalao

Worked example Ex 9 · Ek target p-value given hai, required

$z$ (aur count) dhundo Ek exam poochta hai: " $α = 0.01$ par two-tailed test ke liye, kaun sa critical $∣ z ∣$ observed statistic se exceed karna chahiye? Coin ke liye ( $n = 100$ , $σ = 5$ , $μ = 50$ ), yeh kitne heads se correspond karta hai?"

Forecast: hum p-value formula invert karte hain. Zyada strict standard (0.01 vs 0.05) ka matlab hai humein bada $z$ chahiye.

Significance split karo. Two-tailed $α = 0.01$ har tail mein $0.005$ dalta hai. Yeh step kyun? Critical value wahan hai jahan right tail area exactly $α /2$ ke barabar hoti hai.
Normal invert karo. Humein woh $z^{*}$ chahiye jiska right-tail area exactly $0.005$ ho, yani $P (Z \geq z^{*}) = 0.005$ . Standard normal ko ulta padhna — uska quantile function, jo ek probability leta hai aur woh $z$ return karta hai jo use produce karta hai — $z^{*} = 2.575829$ deta hai. Yeh step kyun? Forward direction ( $z \to$ area) woh hai jo humne poori page par kiya. Yahan hum area jaante hain aur $z$ chahte hain; yahi exactly woh hai jo quantile (inverse-CDF) function return karta hai.
Head count mein convert karo. Standardizing formula reverse karo: $z = (X - μ) / σ \Rightarrow X = μ + z σ$ . Toh $X = μ + z^{*} σ = 50 + 2.575829 (5) = 50 + 12.8791 = 62.8791$ . Yeh step kyun? Humne counts ko $z$ mein turn kiya tha; ab hum ek target $z$ ko wapas count mein turn karte hain taaki answer un units mein ho jo question ne poocha (heads).
Whole count tak round karo. Heads whole numbers mein aate hain, isliye hum boundary $62.8791$ ko upar $63$ tak round karte hain (reject karne ke liye tumhe critical value se strictly exceed karna hoga). Symmetry se left boundary $37$ tak round hoti hai. Yeh step kyun? $62$ ka count $62.8791$ se kum padega; sirf $63$ ya zyada use clear karta hai.
Answer state karo. Koi bhi observed $∣ z ∣ > 2.5758$ — equivalently $X \geq 63$ heads ya $X \leq 37$ heads — $α = 0.01$ par $H_{0}$ reject karta hai.

Verify: $2 \cdot P (Z \geq 2.575829) = 0.01$ exactly ✓, aur $50 + 2.575829 \cdot 5 = 62.8791$ ✓. Thresholds compare karo: $63$ heads strict $α = 0.01$ bar clear karta hai, jabki Ex 1 ka $64$ aasaan $α = 0.05$ bar bhi clear karta hai — stricter $α$ , zyada uunchi hurdle.

Matrix wrap karna

Scenario matrix ka har cell ab ek solved example hai. Nau ke neeche ek single skill:

$z = \frac{observed - null}{standard error}, p-value = (tails) \cdot P (Z beyond z)$

Sirf do judgement calls hain: kitne tails ( $H_{1}$ ki wording se) aur kaun sa side (effect ke sign se). Yeh do sahi karo aur arithmetic har baar identical hai.

Recall Quick self-test

Two-tailed $z = - 2.8$ : p-value kya hai? ::: $2 \cdot P (Z \geq 2.8) = 0.00511$ — sign $∣ z ∣$ mein fold ho jaata hai. One-tailed right test, lekin effect galat way gaya ( $z < 0$ ): kya tum kabhi reject kar sakte ho? ::: Nahi — right-tail p-value $0.5$ se exceed kar jaayega; wrong-direction effect kabhi $H_{1}$ ka evidence nahi hota. Observation exactly null ke barabar hai: $z$ aur p-value kya hai? ::: $z = 0$ , p-value $= 1$ (two-tailed) — maximally boring. $0.5%$ effect $n = 100000$ par $H_{0}$ reject kyun karta hai lekin $n = 200$ par nahi? ::: SE $1/ n$ ki tarah shrink karta hai, isliye $z$ $n$ ke saath grow karta hai chahe effect fixed ho. $π$ , $π_{0}$ , aur $\overset{π}{^}$ mein kya difference hai? ::: $π$ true unknown proportion hai; $π_{0}$ woh value hai jo $H_{0}$ claim karta hai; $\overset{π}{^}$ woh hai jo sample ne dikhaya. Capital $Z$ aur lowercase $z$ mein kya difference hai? ::: $Z$ random variable hai (poori bell); $z$ woh single observed number hai jo hamare data ne produce kiya.

Yeh bhi dekho 1.3.21-Type-I-and-Type-II-errors ("reject" ka kya cost hai jab $H_{0}$ true tha) aur 2.5.7-Statistical-significance-in-experiments (yeh tests real A/B decisions ko kaise power karte hain).