Worked examples — Properties of estimators — unbiasedness, consistency, efficiency

This is a companion drill page for the parent topic. The parent built the three lenses — unbiasedness (right on average), consistency (homes in as $n$ grows), and efficiency (least wobbly). Here we hit every kind of scenario those ideas can throw at you, one worked example per case.

Before we start, four symbols we will reuse. If a line uses them, this is where they were earned:

---

The scenario matrix

Every estimator question lives in one of these cells. The examples below are labelled by the cell they cover, and together they fill the whole grid — Ex 1 through Ex 10 cover C1 through C10 with nothing skipped.

Cell	Case class	What can go wrong / be tested	Example
C1	Unbiased at fixed $n$ (linear estimator)	do the weights sum to 1?	Ex 1
C2	Efficiency comparison (both unbiased)	which variance is smaller, by how much	Ex 2
C3	Optimal weighting (unequal variances)	naive equal weights is NOT best	Ex 3
C4	Degenerate estimator (uses one point)	unbiased but NOT consistent	Ex 4
C5	Biased estimator, boundary/max statistic	sign of bias, unbiased correction	Ex 5
C6	MSE beats unbiasedness (shrinkage)	biased can win on MSE	Ex 6
C7	Limiting behaviour ($n\to \infty$)	consistency of a biased rule	Ex 7
C8	Cramér–Rao efficiency (does it hit the bound?)	Var vs $1/(n{I}_1)$	Ex 8
C9	Real-world word problem	translate story → estimator	Ex 9
C10	Exam twist (combine two rules)	best linear combination	Ex 10

Prerequisite links you may want open while reading: Sampling Distributions, Law of Large Numbers, Fisher Information, Bias–Variance Tradeoff, Maximum Likelihood Estimation, Central Limit Theorem.

---

Example 1 — C1: is a weighted average unbiased?

Forecast: guess now — does it hit $\mu$ on average, or is it tilted? (Hint: look at the weights.)

1. Write the expectation and use linearity. $\mathbb{E}[\hat{\mu }]=0.5\mathbb{E}[X_1]+0.2\mathbb{E}[X_2]+0.3\mathbb{E}[X_3]=(0.5+0.2+0.3)\mu .$ Why this step? Expectation passes through sums and constants (linearity), so a weighted average of the data becomes the same weighted average of the means. Each $\mathbb{E}[X_i]=\mu$.

2. Add the weights. $0.5+0.2+0.3=1\Rightarrow \mathbb{E}[\hat{\mu }]=1\cdot \mu =\mu .$ Why this step? The bias is $(\text{sum of weights}-1)\mu$. It vanishes exactly when the weights sum to 1 — this is the general rule for any linear estimator of the mean.

Verify: if instead the weights were $0.5,0.2,0.4$ (sum $1.1$) we'd get $\mathbb{E}[\hat{\mu }]=1.1\mu \ne \mu$ — biased. Our weights sum to 1, so unbiased. ✓

---

Example 2 — C2: which unbiased estimator is more efficient?

Forecast: which cloud is tighter — the one using all three points, or the one throwing $X_2$ away?

1. Variance of ${\hat{\mu }}_A$. For independent variables, variance of a weighted sum is $\sum w_i^2{\sigma }^2$. $\mathrm{Var}({\hat{\mu }}_A)={\left(\frac{1}{3}\right)}^2{\sigma }^2\cdot 3=\frac{{\sigma }^2}{3}.$ Why this step? Independence kills all covariance cross-terms, so only the $w_i^2$ terms survive.

2. Variance of ${\hat{\mu }}_B$. $\mathrm{Var}({\hat{\mu }}_B)={\left(\frac{1}{2}\right)}^2{\sigma }^2\cdot 2=\frac{{\sigma }^2}{2}.$ Why this step? Same rule, but now only two points each carry weight $\frac{1}{2}$, so two $w_i^2$ terms survive.

3. Relative efficiency.

Verify: ${\sigma }^2/3<{\sigma }^2/2$ for any ${\sigma }^2>0$, so ${\hat{\mu }}_A$ is more efficient. Using all the data pays off. ✓

---

Example 3 — C3: optimal weights when variances differ

Forecast: should you weight the noisy $Y_2$ equally, less, or more than $Y_1$?

1. Unbiasedness fixes the weights to sum to 1. With weights $w$ and $1-w$, $\mathbb{E}[\hat{\mu }]=\mu$ automatically. Good — one constraint handled. Why this step? By Ex 1's rule, weights summing to $1$ guarantee unbiasedness; that pins down the second weight as $1-w$ and leaves one free parameter $w$ to optimise.

2. Write the variance. $V(w)=w^2\cdot 1+(1-w{)}^2\cdot 4.$ Why this step? Independence again → add the two scaled variances.

3. Minimise: differentiate and set to zero. $V^{\prime }(w)=2w-8(1-w)=10w-8=0\Rightarrow w=\frac{8}{10}=\mathrm{0.8.}$ Why the derivative? $V(w)$ is an upward parabola in $w$; its lowest point is where the slope is zero. That is exactly what $V^{\prime }(w)=0$ locates. The general rule: weight each measurement inversely proportional to its variance — here $\frac{1/1}{1/1+1/4}=\frac{1}{1.25}=0.8$. ✓ matches.

4. Resulting minimum variance. $V(0.8)=0.8^2+0.2^2\cdot 4=0.64+0.16=\mathrm{0.8.}$ Why this step? Plug the optimal $w$ back into $V(w)$ to read off the smallest achievable variance.

Verify: equal weights $w=0.5$ give $V=0.25+0.25\cdot 4=1.25>0.8$. Inverse-variance weighting wins. Lesson: equal weights (Ex 2) are optimal only when variances are equal. ✓

---

Example 4 — C4: unbiased but NOT consistent (degenerate rule)

Forecast: it's obviously "silly", but which of the two properties does it actually fail?

1. Unbiased? $\mathbb{E}[X_1]=\mu$. Yes, unbiased for every $n$. Why this step? The first draw has mean $\mu$ regardless of how many other points exist.

2. Consistency check via MSE. Bias $=0$, so $\mathrm{MSE}=\mathrm{Var}(X_1)={\sigma }^2$ — a constant, independent of $n$. Why this step? Consistency needs $\mathrm{MSE}\to 0$. Here it never shrinks; collecting more data does nothing because you throw it away.

3. Conclusion. $\mathrm{MSE}={\sigma }^2\nrightarrow 0\Rightarrow$ not consistent. Why this step? The MSE→0 sufficient condition simply cannot fire for a constant MSE, so the estimator fails the limit test.

Verify: contrast with $\bar{X}$ where $\mathrm{MSE}={\sigma }^2/n\to 0$. Same bias (zero), utterly different limit. This is the concrete counterexample to "unbiased $\Rightarrow$ consistent." ✓

---

Example 5 — C5: boundary/max statistic — sign of the bias

Forecast: the maximum can never exceed $\theta$ — does that pull the estimate above or below the truth?

Figure s01 — one Uniform$(0,\theta )$ sample of size $n=3$ on the value axis: the three points and their maximum all sit strictly below the true upper limit $\theta$, and the plum gap on the right is the downward bias.

1. Distribution of the max. $P(\max \le t)={\left(\frac{t}{\theta }\right)}^n$ for $0\le t\le \theta$. Why this step? All $n$ points must fall below $t$; each does so with probability $t/\theta$, and they're independent, so we multiply.

2. Expectation of the max. Integrating gives the standard result $\mathbb{E}[\max ]=\frac{n}{n+1}\theta .$ Why this step? The density is $\frac{n{t}^{n-1}}{{\theta }^n}$; ${\int }_0^{\theta }t\cdot \frac{n{t}^{n-1}}{{\theta }^n}dt=\frac{n}{n+1}\theta$.

3. Sign of bias. $\mathrm{Bias}=\frac{n}{n+1}\theta -\theta =-\frac{1}{n+1}\theta <0$. The max always underestimates — matching the figure, where the maximum sits inside $[0,\theta ]$ and can only approach $\theta$ from below, never reach it. Why this step? The bias is negative for every $n$, so this is a systematic under-shoot, not random luck.

4. Unbiased fix. Multiply by $\frac{n+1}{n}$: ${\hat{\theta }}_{\text{unb}}=\frac{n+1}{n}{\max }_i{X}_i,\mathbb{E}[{\hat{\theta }}_{\text{unb}}]=\theta .$ Why this step? Scaling up by exactly the reciprocal of $\frac{n}{n+1}$ cancels the shrink, restoring $\mathbb{E}=\theta$.

Verify (n=3): $\mathbb{E}[\max ]=\frac{3}{4}\theta$, bias $=-\frac{1}{4}\theta$. Correction factor $\frac{4}{3}$, and $\frac{4}{3}\cdot \frac{3}{4}\theta =\theta$. ✓

---

Example 6 — C6: a biased estimator that beats the unbiased one on MSE

Forecast: unbiased sounds best — but if we shrink the estimate a little, can trading a touch of bias for less variance lower the total MSE?

We need two facts about $M$ for $n=3,\theta =1$: $\mathbb{E}[M]=\frac{3}{4}$ and $\mathbb{E}[M^2]=\frac{n}{n+2}=\frac{3}{5}$, so $\mathrm{Var}(M)=\frac{3}{5}-\frac{9}{16}=\frac{3}{80}$.

1. MSE of a general ${\hat{\theta }}_c=cM$. With bias $c\mathbb{E}[M]-1=\frac{3}{4}c-1$ and variance $c^2\mathrm{Var}(M)$, $\mathrm{MSE}(c)=c^2\mathbb{E}[M^2]-2c\mathbb{E}[M]+1=\frac{3}{5}{c}^2-\frac{3}{2}c+1.$ Why this step? $\mathrm{MSE}=\mathbb{E}[(cM-1{)}^2]=c^2\mathbb{E}[M^2]-2c\mathbb{E}[M]+1$ — just expand the square; this bundles Var and Bias${}^2$ into one clean quadratic in $c$.

2. Minimise over $c$. Differentiate and set to zero: ${\mathrm{MSE}}^{\prime }(c)=\frac{6}{5}c-\frac{3}{2}=0\Rightarrow c^{\star }=\frac{3/2}{6/5}=\frac{5}{4}=\mathrm{1.25.}$ Why the derivative? $\mathrm{MSE}(c)$ is an upward parabola in $c$; its minimum is where the slope is zero. The MSE-optimal factor $c^{\star }=\frac{5}{4}$ is smaller than the unbiased factor $\frac{4}{3}\approx 1.33$ — it deliberately shrinks the estimate, accepting a little bias.

3. MSE at the optimal (biased) rule. $\mathrm{MSE}(\frac{5}{4})=\frac{3}{5}\cdot \frac{25}{16}-\frac{3}{2}\cdot \frac{5}{4}+1=\frac{15}{16}-\frac{15}{8}+1=\frac{1}{16}=\mathrm{0.0625.}$ Why this step? Plug $c^{\star }$ back into the quadratic to read off the smallest attainable MSE in this family.

4. MSE of the unbiased $\frac{4}{3}M$. Bias $=0$, so $\mathrm{MSE}(\frac{4}{3})={\left(\frac{4}{3}\right)}^2\mathrm{Var}(M)=\frac{16}{9}\cdot \frac{3}{80}=\frac{1}{15}\approx \mathrm{0.0667.}$ Why this step? When bias is zero the MSE is just the variance, so scale $\mathrm{Var}(M)$ by the squared factor.

5. Compare. $0.0625<0.0667$: the biased shrunk rule $c^{\star }=\frac{5}{4}$ has strictly lower MSE than the unbiased rule. Why show this? This is the honest headline of the Bias–Variance Tradeoff: "unbiased" is not the same as "best." A little deliberate bias can buy enough variance reduction to lower the total squared error.

Verify: all three MSE numbers ($c=\frac{5}{4}\to \frac{1}{16}$, $c=\frac{4}{3}\to \frac{1}{15}$, and $\frac{1}{16}<\frac{1}{15}$) checked below. ✓

---

Example 7 — C7: a biased rule that is still consistent

First, two objects this example leans on — define them before use:

Forecast: biased at every finite $n$ — does the bias survive the limit, or wash out?

1. The bias. From the parent, $\mathbb{E}[\sum (X_i-\bar{X}{)}^2]=(n-1){\sigma }^2$. Dividing by $n$: $\mathbb{E}[{\hat{\sigma }}_n^2]=\frac{n-1}{n}{\sigma }^2,\mathrm{Bias}=\frac{n-1}{n}{\sigma }^2-{\sigma }^2=-\frac{{\sigma }^2}{n}.$ Why this step? Centering on $\bar{X}$ instead of $\mu$ shrinks the deviations, costing one degree of freedom — the classic $n-1$ story.

2. The variance shrinks too. Write ${\hat{\sigma }}_n^2=\frac{n-1}{n}{s}^2$ where $s^2$ is the unbiased sample variance just defined. Then $\mathrm{Var}({\hat{\sigma }}_n^2)={\left(\frac{n-1}{n}\right)}^2\mathrm{Var}(s^2),\mathrm{Var}(s^2)=\frac{1}{n}\left({\mu }_4-\frac{n-3}{n-1}{\sigma }^4\right).$ Why this step? The standard formula for $\mathrm{Var}(s^2)$ has an explicit $\frac{1}{n}$ out front (it needs a finite fourth central moment ${\mu }_4$). So $\mathrm{Var}(s^2)\to 0$ like $1/n$, and multiplying by ${\left(\frac{n-1}{n}\right)}^2\to 1$ leaves $\mathrm{Var}({\hat{\sigma }}_n^2)\to 0$ as well. Both pieces of the MSE vanish.

3. Limit of the bias. $-{\sigma }^2/n\to 0$ as $n\to \infty$. Why this step? We now have both ingredients of $\mathrm{MSE}=\mathrm{Var}+{\mathrm{Bias}}^2$ going to zero.

4. Conclusion: MSE → 0 ⇒ consistent. Since $\mathrm{Var}({\hat{\sigma }}_n^2)\to 0$ (step 2) and ${\mathrm{Bias}}^2={\sigma }^4/n^2\to 0$ (step 3), $\mathrm{MSE}({\hat{\sigma }}_n^2)=\mathrm{Var}+{\mathrm{Bias}}^2\to 0\Rightarrow {\hat{\sigma }}_n^2\text{ is }\mathbf{consistent}.$ Why this step? The MSE→0 sufficient condition fires: even though the estimator is biased at every finite $n$, the bias vanishes in the limit, so consistency holds.

Verify (n=10, σ²=5): $\mathbb{E}[{\hat{\sigma }}_{10}^2]=\frac{9}{10}\cdot 5=4.5$, bias $=-0.5$. As $n\to \infty$ the factor $\frac{n-1}{n}\to 1$ and $\mathrm{Var}\to 0$. ✓ Biased ≠ inconsistent.

---

Example 8 — C8: does an estimator hit the Cramér–Rao bound?

Forecast: we already know $\mathrm{Var}(\bar{X})={\sigma }^2/n$. Will the information bound land on exactly that, or leave room to improve?

1. Fisher information per observation. The log-density is $\ln f=-\frac{(x-\mu {)}^2}{2{\sigma }^2}+\text{const}$. Its $\mu$-derivative (the score): $\frac{\partial }{\partial \mu }\ln f=\frac{x-\mu }{{\sigma }^2}.$ Why this tool? Fisher Information is the expected squared score — it measures how sharply the likelihood reacts to $\mu$. A sharp reaction means the data pin $\mu$ down, so the achievable variance is small.

2. Square and take expectation. $I_1(\mu )=\mathbb{E}\left[{\left(\frac{X-\mu }{{\sigma }^2}\right)}^2\right]=\frac{\mathbb{E}[(X-\mu {)}^2]}{{\sigma }^4}=\frac{{\sigma }^2}{{\sigma }^4}=\frac{1}{{\sigma }^2}.$ Why this step? $\mathbb{E}[(X-\mu {)}^2]={\sigma }^2$ by definition of variance, so the ratio collapses to $1/{\sigma }^2$.

3. The CRLB for $n$ i.i.d. points. $\mathrm{Var}(\hat{\mu })\ge \frac{1}{n{I}_1(\mu )}=\frac{1}{n\cdot (1/{\sigma }^2)}=\frac{{\sigma }^2}{n}.$ Why this step? Information adds across independent observations, so total information is $n{I}_1$, and the CRLB is its reciprocal.

4. Compare with $\bar{X}$. $\mathrm{Var}(\bar{X})={\sigma }^2/n$ = the bound exactly. Why this step? Equality with the CRLB is the definition of an efficient estimator.

Verify: bound $={\sigma }^2/n=$ actual variance $\Rightarrow$ $\bar{X}$ is efficient (the MVUE for a known-variance normal mean). ✓

---

Example 9 — C9: real-world word problem

Forecast: guess how much more precise measuring $256$ bolts is versus $64$ — double? quadruple?

1. (a) Properties. $\bar{X}$ is unbiased ($\mathbb{E}[\bar{X}]=\mu$), consistent ($\mathrm{Var}={\sigma }^2/n\to 0$), and efficient for a normal mean (Ex 8). So it is the best simple choice, and the reported estimate is $50.1$ mm. Why this step? We translate the story into the estimator $\bar{X}$ and check all three lenses before trusting the number.

2. (b) Standard error at $n=64$. The standard error is $\mathrm{SD}(\bar{X})=\sigma /\sqrt{n}$. $\frac{2}{\sqrt{64}}=\frac{2}{8}=0.25\text{ mm}.$ Why this step? Variance falls like $1/n$, so the standard deviation (its square root) falls like $1/\sqrt{n}$. Units: mm, matching $\mu$.

3. Standard error at $n=256$. $\frac{2}{\sqrt{256}}=\frac{2}{16}=0.125\text{ mm}.$ Why this step? Same formula $\sigma /\sqrt{n}$ with the new sample size; the larger sample gives a smaller number, i.e. a tighter estimate.

4. How much did it shrink? Take the ratio of the two standard errors: $\frac{\text{SE}(64)}{\text{SE}(256)}=\frac{0.25}{0.125}=2.$ Why this step? Quadrupling $n$ (from $64$ to $256$) multiplies $\sqrt{n}$ by $\sqrt{4}=2$, so the standard error is halved — not quartered. Precision improves only as $\sqrt{n}$, the diminishing return baked into every mean estimator (this is Law of Large Numbers with a rate set by the Central Limit Theorem).

Verify: SE at $64=0.25$ mm, at $256=0.125$ mm, ratio $=2$. ✓ Units mm throughout, and both values are positive and shrinking as expected.

---

Example 10 — C10: exam twist — best combination of two estimators

Forecast: since ${\hat{\mu }}_1$ is already the tighter one, will the best $w$ be above $0.5$? And can combining beat ${\hat{\mu }}_1$ alone?

1. Inverse-variance weighting. Same machinery as Ex 3: optimal $w=\frac{1/2}{1/2+1/6}$. $w=\frac{1/2}{1/2+1/6}=\frac{1/2}{2/3}=\frac{3}{4}=\mathrm{0.75.}$ Why this step? Minimising $w^2(2)+(1-w{)}^2(6)$ via the derivative gives $w=\frac{V_2}{V_1+V_2}=\frac{6}{8}=0.75$ — equivalently inverse-variance weights.

2. Minimum variance. $V_{\min }=\frac{1}{1/V_1+1/V_2}=\frac{1}{1/2+1/6}=\frac{1}{2/3}=\mathrm{1.5.}$ Why this step? For inverse-variance weighting of independent unbiased estimators, the combined precision (reciprocal variance) is the sum of the individual precisions.

3. Relative efficiency vs ${\hat{\mu }}_1$ alone (better over worse). $\frac{\mathrm{Var}({\hat{\mu }}_1)}{V_{\min }}=\frac{2}{1.5}=\frac{4}{3}\approx \mathrm{1.33.}$ Why this step? Combining even a noisier estimator strictly lowers variance (from $2$ to $1.5$), so the combination is the tighter one; putting its variance in the denominator gives a ratio $>1$.

Verify: plug $w=0.75$: $0.75^2\cdot 2+0.25^2\cdot 6=1.125+0.375=1.5$ ✓, matching the closed form. Relative efficiency $4/3$. ✓

---

Recall Quick self-test (cover the answers)

Unbiased but not consistent — name the rule ::: $\hat{\mu }=X_1$ (variance never shrinks). Optimal weight for independent estimators is proportional to ::: the reciprocal of each variance (inverse-variance weighting). Standard error of $\bar{X}$ scales like ::: $1/\sqrt{n}$ — quadruple $n$ to halve it. $\bar{X}$ hits the Cramér–Rao bound for a normal mean because ::: its variance ${\sigma }^2/n$ equals $1/(n{I}_1)$ with $I_1=1/{\sigma }^2$. The max of a Uniform$(0,\theta )$ sample is biased ::: low (always underestimates), fixed by the factor $\frac{n+1}{n}$; the MSE-optimal shrinkage factor is even smaller, $\frac{5}{4}$ at $n=3$.