Hypothesis testing — null - alternative, test statistic, p-value, errors (Type I & II): Question bank

4.9.20 · D5 · Maths › Probability Theory & Statistics › Hypothesis testing — null - alternative, test statistic, p-v

Un words ke reminders jo hum use karte hain, sab parent mein define hain:

$H_{0}$ = woh boring "no effect" claim jo hum true maanke chalte hain; $H_{1}$ = woh claim jiske liye hum evidence dhundhte hain.
$α$ = woh false-alarm rate jo hum pehle se choose karte hain; $p$ = yeh data kitna surprising hai agar $H_{0}$ hold kare.
Type I = ek sach wale $H_{0}$ ko reject karna; Type II = ek jhoothe $H_{0}$ ko reject karne mein fail hona; power $= 1 - β$ .

True or false — justify

A small p-value proves the null hypothesis is false.

False. Ek chhota

p

sirf yeh kehta hai ki data surprising hai agar $H_{0}$ hold kare; yeh

P (data ∣ H_{0})

hai, na ki

P (H_{0} ∣ data)

, isliye yeh kabhi

H_{0}

ko "prove" nahi kar sakta ki woh false hai — ek sache

H_{0}

ke under ek rare fluke exactly

α

fraction of the time chhota

p

produce karta hai.

p = 0.20

, there is a

20%

chance the null is true.

False.

p

ko compute karte waqt

H_{0}

ko pehle se true maana jaata hai; yeh

H_{0}

ki probability ke baare mein kuch nahi kehta.

P (data ∣ H_{0})

P (H_{0})

mein convert karna prosecutor's fallacy hai —

P (H_{0} ∣ data)

ke baare mein baat karne ke liye tumhe Bayesian Inference aur ek prior chahiye hoga.

A result with

p = 0.049

is meaningfully stronger evidence than one with

p = 0.051

Kisi bhi real sense mein False.

α = 0.05

ki line ek chosen convention hai, nature ka koi kanoon nahi; dono data sets almost identical hain, isliye ek ko "significant" aur doosre ko "not" treat karna ek hard threshold ka artefact hai.

Failing to reject

H_{0}

means we have proven

H_{0}

is true.

False. "Fail to reject" ka matlab hai ki evidence convict karne ke liye kaafi strong nahi tha — aksar sirf chhota

n

ya low power hota hai. Evidence ki absensi, absence of evidence nahi hai; hum kehte hain "insufficient evidence," kabhi "proven" nahi.

Lowering

α

from

0.05

0.01

makes the test better in every way.

False. Ek stricter

α

critical cutoff ko aur dur push karta hai, false alarms ghataata hai lekin

β

(Type II errors) badhaata hai, isliye tum zyada real effects miss karte ho. Yeh ek trade hai, free upgrade nahi — sirf bada

n

dono ko improve karta hai.

The alternative hypothesis

H_{1}

can contain the equality (e.g.

μ = μ_{0}

False.

H_{0}

hamesha equality carry karta hai kyunki sampling distribution aur p-value compute karne ke liye humein ek specific value chahiye.

H_{1}

states

\neq =

>

, ya

<

Power

1 - β

depends on how far the true mean actually is from

μ_{0}

True. Real effect jitna bada hoga, alternative sampling distribution

H_{0}

se utni hi dur shift hogi, isliye uska zyada hissa critical cutoff ke past land karta hai aur power badhti hai. Dekho Statistical Power & Sample Size.

Increasing sample size

n

shrinks both

α

and

β

α

ke liye False,

β

ke liye True.

α

ek value hai jo hum choose karte hain, isliye

n

usse touch nahi karta; lekin bada

n

Standard Error ko shrink karta hai, dono distributions ko narrow karta hai isliye same

α

par

β

drop ho jaata hai.

95%

confidence interval and a two-tailed test at

α = 0.05

always agree about

μ_{0}

True.

α = 0.05

par

H_{0} : μ = μ_{0}

ko reject karna equivalent hai

μ_{0}

95%

confidence interval ke bahar girne se — yeh ek hi standardized distance ke do views hain.

Spot the error

"We got

p = 0.03 < 0.05

, so there's a

97%

chance our effect is real."

Error yeh hai ki

1 - p

ko probability maan liya ki effect exist karta hai.

p

H_{0}

par conditional hai; effect real hone ki probability ke liye ek prior aur Bayesian machinery chahiye.

H_{0}

ko reject karna ek decision hai, reality ke baare mein koi probability statement nahi.

"Our test wasn't significant, so the drug has no effect."

"Fail to reject" ko "proven no effect" se confuse kar raha hai. Ek null result sirf low power ya chhote

n

ko reflect kar sakta hai; sach mein effect real ho sakta hai lekin itna chhota ki is test ke liye catchable na ho.

α = 0.05

means at most

5%

of all my published rejections are false discoveries."

α = P (reject ∣ H_{0} true)

— null ke true hone par conditional hai. Tumhare rejections mein se jo galat hain unka share (false discovery rate) yeh bhi depend karta hai ki

H_{0}

actually kitni baar true hota hai, jo

α

akele kabhi nahi batata.

"The data leaned high, so I'll switch to a right-tailed test to get significance."

Data dekhne ke baad tail choose karna tumhara real false-alarm rate double kar deta hai — tum effectively dono directions test karte ho jabki ek claim karte ho. Direction data dekhne se pehle fix hona chahiye.

"We used

Z = (\overset{ˉ}{X} - μ_{0}) / (s / n)

with a sample of

n = 9

and treated it as standard normal."

Jab

σ

unknown ho aur

s

se estimate kiya jaaye, statistic

n - 1

degrees of freedom ke saath Student t-distribution follow karta hai, na ki

N (0, 1)

. Chhote

n

ke liye uski heavier tails matter karti hain;

z

use karna p-value ko understate karta hai.

"We ran

20

separate tests at

α = 0.05

and one came out significant, so we found something."

20

independent true nulls ke saath tum chance se expect karte ho lagbhag ek false positive (

20 \times 0.05 = 1

). Bahut saare tests mein se ek hit exactly false-alarm rate ka kaam hai, koi discovery nahi — tumhe ek multiple-comparison correction chahiye.

"Our huge sample gave

p = 0.001

, so the effect is large and important."

Ek massive

n

se aaya chhota p-value ek trivially small effect ko flag kar sakta hai — statistical significance, practical significance nahi hai. Bada

n

standard error shrink karta hai isliye

μ_{0}

se ek microscopic difference bhi "detectable" ban jaata hai.

Why questions

Why must

H_{0}

always contain the equality rather than an inequality?

Kyunki humein ek exact value (

μ = μ_{0}

) chahiye taaki ek specific sampling distribution pin down ho sake aur p-value compute ho sake;

μ > μ_{0}

jaisi inequality infinitely many distributions name karti hai aur humein standardize karne ke liye kuch nahi deti.

Why do we double the tail probability for a two-tailed p-value?

Ek two-sided test mein "as or more extreme" ka matlab hai kisi bhi direction mein door hona, isliye dono symmetric tails evidence count hoti hain — hence

p = 2 P (Z \geq ∣ z_{o b s} ∣)

Why does the test statistic divide by the standard error instead of the standard deviation

σ

Hum judge kar rahe hain ki sample mean

\overset{ˉ}{X}

μ_{0}

se kitna dur hai, aur

\overset{ˉ}{X}

spread

σ / n

ke saath vary karta hai (Standard Error), na ki

σ

ke saath. SE se divide karna ek unit-free count deta hai ki "kitne standard errors door."

Why does the Central Limit Theorem let us use the normal distribution even when the raw data isn't normal?

CLT kehta hai ki

\overset{ˉ}{X}

ki sampling distribution

n

badhne ke saath normal ke paas jaati hai, chahe population ka shape kuch bhi ho — isliye standardized statistic approximately standard normal hai even skewed data ke liye.

Why can't we ever "accept" the null hypothesis?

Kyunki test sirf deviations detect karne ke liye design kiya gaya hai, unki absence confirm karne ke liye nahi; ek non-significant result

H_{0}

ke saath aur bahut saare chhote real effects ke saath consistent hai jinhe hum dekhne mein powerless the. Hum sirf itna keh sakte hain ki reject karne ke liye kaafi evidence nahi mila.

Why is a Type I error controlled directly but a Type II error is not?

Hum

α

ko cutoff ke roop mein choose karte hain, isliye Type I rate hum design se set karte hain.

β

unknown true effect size,

n

, aur

α

sab par ek saath depend karta hai, isliye yeh emerge hota hai, directly dial-in nahi kiya jaata.

Why does making a test more powerful usually require more data rather than just a lower

α

α

ghataana actually power reduce karta hai. Power do sampling distributions ko alag karke aur unhe narrow karke badhti hai — bada

n

standard error shrink karta hai aur dono kaam karta hai, jabki

α

sirf cutoff move karta hai.

Edge cases

If the observed sample mean lands exactly on

μ_{0}

, what is the two-tailed p-value?

z_{o b s} = 0

, isliye

p = 2 P (Z \geq 0) = 2 (0.5) = 1

. Data

H_{0}

ke under least surprising possible hai — tum kabhi reject nahi karte.

What happens to the t-statistic's distribution as

n \to \infty

n - 1

degrees of freedom ke saath Student t-distribution standard normal ke paas converge hoti hai, kyunki

s

σ

ka near-perfect estimate ban jaata hai aur extra tail-uncertainty gayab ho jaati hai.

If you set

α = 0

to eliminate all false alarms, what happens to your ability to detect real effects?

Critical cutoff infinity par chali jaati hai, isliye tum

H_{0}

kabhi reject nahi kar sakte — Type I error zero hai lekin

β = 1

aur power zero hai. Tumne ek aisa test banaya hai jo hamesha acquit karta hai, kuch bhi dhundhne ke liye useless.

If you set

α = 1

, what does the test do?

Tum kisi bhi data ke liye

H_{0}

reject karte ho; power

1

hai lekin har sach wala null falsely reject hota hai. Dono extremes (

α = 0

aur

α = 1

) dikhate hain kyun

α

ek deliberate compromise hai.

For a one-sided right-tailed test, what if the sample mean falls below

μ_{0}

Evidence

H_{1} : μ > μ_{0}

ke opposite direction mein point karta hai, isliye

z_{o b s} < 0

aur

p = P (Z \geq z_{o b s}) > 0.5

. Tum reject nahi kar sakte — jo result tumhare alternative ke against jaata hai woh kabhi usse support nahi kar sakta.

What if two different analysts pick

α = 0.05

and

α = 0.01

on the same data with

p = 0.03

Pehla

H_{0}

reject karta hai, doosra reject karne mein fail hota hai — dono sahi hain apni chosen risk tolerance ke hisaab se. P-value data se fixed hai; verdict us threshold par depend karta hai jo har ek ne pehle choose kiya.

As the true effect size shrinks toward zero, what happens to the power of a fixed test?

Power

α

ki taraf girti hai. Jab essentially koi effect nahi hota, alternative distribution

H_{0}

ke almost upar baith jaati hai, isliye sirf wahi rejections hote hain jo

α

rate par false alarms hain.

Recall One-line self-check

Upar ke har answer ko cover karo; agar koi verdict sirf bare "true/false" ke roop mein aaya bina kisi reason ke, tum use earn nahi kiya — reasoning hi concept hai.