Hypothesis testing — null - alternative, test statistic, p-value, errors (Type I & II): Exercises

This page is a self-testing ladder. Work each problem before opening its solution. Levels climb from "spot the pieces" to "design a whole test." The parent note Hypothesis testing — null - alternative, test statistic, p-value, errors (Type I & II) builds the theory; the prerequisites Central Limit Theorem, Standard Error, and Normal Distribution build the tools. But you should not need to leave this page — every symbol is re-anchored below before use.

Definition The symbols on this page, in plain words

$μ_{0}$ — the claimed population mean under $H_{0}$ (the "boring" value we assume true).
$\overset{ˉ}{X}$ — the sample mean, the average of the numbers we actually measured.
$σ$ — the true population standard deviation (spread), a known fixed number.
$s$ — the sample standard deviation, our estimate of $σ$ from the data.
$n$ — the sample size, how many observations we collected.
$SE$ — the standard error $= σ / n$ (or $s / n$ ): how much $\overset{ˉ}{X}$ itself wobbles from sample to sample. Bigger $n$ → smaller wobble.
$Z$ — the z-statistic $= \frac{X ˉ - μ _{0}}{SE}$ : how many standard errors $\overset{ˉ}{X}$ sits away from $μ_{0}$ . Under $H_{0}$ it follows a Normal Distribution with mean $0$ , spread $1$ , written $N (0, 1)$ .
$T$ — the t-statistic, same shape as $Z$ but with $s$ in place of $σ$ ; it follows a Student t-distribution $t_{n - 1}$ with fatter tails.
$α$ — the significance level, the Type I error rate we accept (usually $0.05$ ).
$P (Z \geq z)$ — the right-tail area: the probability a standard normal lands at or above $z$ .

A reference table of standard-normal tail areas we reuse (these are $P (Z \geq z)$ , the shaded area to the right of $z$ under the bell curve):

Level 1 — Recognition

Exercise 1.1

For each claim, write $H_{0}$ and $H_{1}$ , and state whether the test is two-, left-, or right-tailed. (a) "A drug changes average blood pressure from the known baseline $μ_{0} = 120$ ." (b) "A new fertilizer increases mean yield above $μ_{0} = 50$ kg." (c) "A factory's defect rate is below the old mean $μ_{0} = 0.03$ ."

Recall Solution 1.1

Remember: == $H_{0}$ always holds the equality== (why? the test needs one precise value to compute probabilities from — an inequality gives no single number to standardize against). The direction word in the claim picks the tail.

(a) "changes" = could go either way → $H_{0} : μ = 120$ , $H_{1} : μ \neq = 120$ → two-tailed.
(b) "increases / above" → $H_{0} : μ = 50$ , $H_{1} : μ > 50$ → right-tailed.
(c) "below" → $H_{0} : μ = 0.03$ , $H_{1} : μ < 0.03$ → left-tailed.

Exercise 1.2

A study reports $p = 0.02$ at significance level $α = 0.05$ . State the decision and phrase it correctly.

Recall Solution 1.2

Rule: reject $H_{0}$ when $p \leq α$ . Here $0.02 \leq 0.05$ → reject $H_{0}$ . Why this rule? $p$ is the chance of data this extreme if $H_{0}$ were true; $α$ is the fluke-rate we tolerate. When $p \leq α$ the data are rarer than our tolerance, so blaming pure luck stops being credible. Correct phrasing: "The data are inconsistent with $H_{0}$ at the $5%$ level; we have evidence for $H_{1}$ ." We do not say " $H_{1}$ is proven."

Level 2 — Application

Exercise 2.1

A coffee machine should pour $μ_{0} = 250$ ml with known $σ = 12$ ml. A sample of $n = 36$ cups gives $\overset{ˉ}{X} = 245$ ml. Test at $α = 0.05$ whether the machine is mis-set.

Recall Solution 2.1

$H_{0} : μ = 250$ , $H_{1} : μ \neq = 250$ (two-tailed — "mis-set" = either direction). Standard error $= σ / n$ . Why divide by $n$ ? Averaging $n$ noisy readings cancels much of the noise; the wobble of $\overset{ˉ}{X}$ shrinks like $1/ n$ . Here $SE = 12/ 36 = 12/6 = 2$ . Standardize: $z = \frac{X ˉ - μ _{0}}{SE} = \frac{245 - 250}{2} = - 2.5$ . Why? This converts " $5$ ml low" into " $2.5$ standard errors low," a unit-free score we can look up. Two-tailed p-value: $p = 2 P (Z \geq 2.5)$ . Why double the one-sided tail? "Mis-set" counts a mean this far above $250$ as equally surprising as this far below. So "as extreme as $z = - 2.5$ " means $∣ Z ∣ \geq 2.5$ , i.e. the tail below $- 2.5$ plus the mirror tail above $+ 2.5$ . Because the normal curve is symmetric, those two tails have equal area, so we take one tail and double it: $p = 2 P (Z \geq 2.5) = 2 (0.00621) = 0.01242.$ $0.01242 \leq 0.05$ → reject $H_{0}$ : the machine is mis-set (pouring low).

Exercise 2.2

A teacher claims a new method raises the mean score above $μ_{0} = 65$ . Sample: $n = 25$ , $\overset{ˉ}{X} = 68$ , sample SD $s = 10$ . Test at $α = 0.05$ .

Recall Solution 2.2

$σ$ is unknown (only the sample $s$ is given) → use the Student t-distribution statistic $T$ . $H_{0} : μ = 65$ , $H_{1} : μ > 65$ (right-tailed — "raises above"). $SE = s / n = 10/ 25 = 10/5 = 2$ . $T = \frac{68 - 65}{2} = 1.5$ with $df = n - 1 = 24$ . Why $df = n - 1$ ? Estimating $s$ "uses up" one degree of freedom (the deviations from $\overset{ˉ}{X}$ must sum to zero), leaving $n - 1$ free. Critical value $t_{0.05, 24} \approx 1.711$ . Since $1.5 < 1.711$ → fail to reject $H_{0}$ . Wording: not enough evidence that the method raises scores.

Level 3 — Analysis

Exercise 3.1

Two students test the same data ( $μ_{0} = 100$ , $σ$ known, $z_{o b s} = 1.8$ ). Student A runs a two-tailed test, Student B a right-tailed test, both at $α = 0.05$ . Compute both p-values and explain why they disagree on the decision.

Recall Solution 3.1

From the table, $P (Z \geq 1.8) = 0.0359$ .

A (two-tailed): $p = 2 (0.0359) = 0.0718 > 0.05$ → fail to reject.
B (right-tailed): $p = 0.0359 \leq 0.05$ → reject. Same number, opposite verdicts. Why? The two-tailed test spends $α$ on both sides ( $2.5%$ each), so it demands a more extreme statistic before rejecting; the one-tailed test pours all $5%$ into one side. This is exactly why the tail must be chosen before seeing data — otherwise you'd pick B because it gives you the result you want.

Exercise 3.2

In Exercise 2.1, describe in words what a Type I and a Type II error would mean for that machine, and say which one $α = 0.05$ directly controls.

Recall Solution 3.2

Type I (false alarm, prob $α$ ): the machine is actually pouring exactly $250$ ml, but we declare it mis-set and recalibrate a perfectly good machine.
Type II (miss, prob $β$ ): the machine really is off, but our sample happened to look normal, so we leave it broken. $α = 0.05$ directly controls the Type I rate: $P (reject ∣ H_{0} true) = 0.05$ . It says nothing on its own about $β$ — that depends on the true $μ$ and on $n$ (see Statistical Power & Sample Size).

Level 4 — Synthesis

Exercise 4.1

A quality engineer will reject a batch if the sample mean is far from $μ_{0} = 500$ . With $σ = 8$ , $n = 64$ (so $SE = 1$ ), she uses a two-tailed test at $α = 0.05$ , rejecting when $∣ z ∣ \geq 1.96$ . (a) Convert the rejection rule into a rule on $\overset{ˉ}{X}$ (the two critical means). (b) Suppose the machine's true mean has drifted to $μ = 502$ . Find the power = probability she correctly rejects. (This is the shaded story in the figure below.)

Figure — how to read it. The figure draws two bell curves on the same $\overset{ˉ}{X}$ -axis. The violet curve is $H_{0}$ (centred at $μ_{0} = 500$ ), the magenta curve is the true distribution of $\overset{ˉ}{X}$ under the drift ( $H_{1}$ , centred at $502$ ). The two navy dashed lines are the critical means $498.04$ and $501.96$ — outside them we reject. The orange shading is the slice of the magenta ( $H_{1}$ ) curve that lies in the rejection region: its total area is the power. Notice almost all of the orange sits in the right tail, because $502$ is close to the upper cutoff and far from the lower one.

Figure — Hypothesis testing — null - alternative, test statistic, p-value, errors (Type I & II)

Recall Solution 4.1

(a) Rejection boundary in $z$ : $z = \pm 1.96$ . Convert with $\overset{ˉ}{X} = μ_{0} + z \cdot SE$ : $\overset{ˉ}{X} = 500 \pm 1.96 (1) = {498.04, 501.96} .$ So she rejects if $\overset{ˉ}{X} \leq 498.04$ or $\overset{ˉ}{X} \geq 501.96$ . (b) If the truth is $μ = 502$ , then $\overset{ˉ}{X} \sim N (502, 1^{2})$ — so we now standardize using $502$ , not $μ_{0}$ . Power $= P (reject ∣ μ = 502)$ = area of the magenta curve past each cutoff:

Upper tail: $P (\overset{ˉ}{X} \geq 501.96) = P (Z \geq \frac{501.96 - 502}{1}) = P (Z \geq - 0.04)$ . Why $P (Z \geq - 0.04) = 0.5160$ ? By the symmetry of the standard normal, area right of $- 0.04$ = (area right of $0$ ) $+$ (area between $0$ and $0.04$ ) $= 0.5 + 0.0160 = 0.5160$ . Equivalently $P (Z \geq - 0.04) = 1 - P (Z \geq 0.04) = 1 - 0.4840 = 0.5160$ .
Lower tail: $P (\overset{ˉ}{X} \leq 498.04) = P (Z \leq \frac{498.04 - 502}{1}) = P (Z \leq - 3.96) \approx 0.0000$ (essentially no area that far out).

Power $\approx 0.5160 + 0.0000 = 0.516$ . Interpretation: even with a real $2$ -ml drift, she catches it only ~ $52%$ of the time. To do better she must raise $n$ (narrows both curves) — the Statistical Power & Sample Size tradeoff, resting on the Central Limit Theorem.

Level 5 — Mastery

Exercise 5.1

You must design a two-tailed $z$ -test of $H_{0} : μ = μ_{0}$ with $σ = 10$ and $α = 0.05$ . You want power $\geq 0.90$ to detect a true difference of $∣ μ - μ_{0} ∣ = 4$ . Find the minimum sample size $n$ . (Use the design formula $n = (\frac{( z _{α /2} + z _{β} ) σ}{μ - μ _{0}})^{2}$ , with $z_{α /2} = 1.96$ for $α = 0.05$ and $z_{β} = 1.2816$ for power $0.90$ .)

Recall Solution 5.1

Plug in $z_{α /2} = 1.96$ , $z_{β} = 1.2816$ , $σ = 10$ , effect $= 4$ : $n = (\frac{( 1.96 + 1.2816 ) \cdot 10}{4})^{2} = (\frac{3.2416 \cdot 10}{4})^{2} = (\frac{32.416}{4})^{2} = (8.104)^{2} = 65.67.$ Sample size must be a whole number and we round up (never down — rounding down loses power): $n = 66.$ Why this formula? It stitches two requirements onto one axis: the cutoff must sit $z_{α /2}$ SEs from $μ_{0}$ (controls $α$ ) and $z_{β}$ SEs from the true mean $μ_{1}$ on the other side (controls $β$ ). The gap between the means is $∣ μ - μ_{0} ∣$ ; setting it equal to $(z_{α /2} + z_{β}) \cdot SE$ and solving $SE = σ / n$ for $n$ gives the box. See Statistical Power & Sample Size and Standard Error.

Exercise 5.2

Explain, in one clean paragraph, why increasing $n$ is the only lever that shrinks both $α$ and $β$ once $α$ is held fixed, while adjusting the cutoff merely trades one for the other.

Recall Solution 5.2

First, clarify the two regimes so we don't conflate them. Regime 1 — fixed cutoff (in raw $\overset{ˉ}{X}$ units): slide the cutoff toward $H_{1}$ and the $H_{0}$ tail past it shrinks ( $α ↓$ ) while the $H_{1}$ tail on the wrong side grows ( $β ↑$ ): a pure trade, no free lunch. Regime 2 — fixed $α$ (the normal practice): we pin $α = 0.05$ , which pins the cutoff at $z_{α /2}$ standard errors from $μ_{0}$ , so $α$ is now a constant by construction — the only question left is how big $β$ is. Here is the payoff of increasing $n$ : because $SE = σ / n$ collapses as $1/ n$ , both bell curves get narrower and, measured in SE units, pull apart. With the $α$ -cutoff held at its fixed $z$ -position, the narrower, better-separated $H_{1}$ curve now hangs almost entirely inside the rejection region, so $β$ falls while $α$ stays exactly $0.05$ . That is the sense in which $n$ shrinks "both": it holds $α$ where you set it and drives $β$ down — something no cutoff move can do. The Central Limit Theorem guarantees $\overset{ˉ}{X}$ stays normal while its spread collapses, so the whole picture keeps working.

Recall One-line self-check on the whole ladder

Did you (i) fix each tail before computing, (ii) use $t$ whenever $σ$ was estimated, (iii) standardize power under $H_{1}$ not $H_{0}$ , (iv) round sample sizes up, and (v) check the normality/CLT assumption before trusting any of it? ::: If yes to all five, you've cleared the classic traps L1–L5.