Properties of estimators — unbiasedness, consistency, efficiency: Visual walkthrough

We are going to prove:

s^{2} = \frac{1}{n - 1} i = 1 \sum n (X_{i} - \overset{ˉ}{X})^{2} has E [s^{2}] = σ^{2} .

Every symbol will be earned before it is used.

Step 1 — The dots, the true centre, and the guessed centre

WHAT. Picture $n$ measurements as dots on a number line. Call them $X_{1}, \dots, X_{n}$ (the little subscript just numbers the dots: $X_{3}$ is "the third dot"). There is a hidden true centre of the population, written $μ$ (Greek "mu", it just means the real average we cannot see). From the dots we compute a guessed centre, the sample mean:

\overset{ˉ}{X} = \frac{1}{n} i = 1 \sum n X_{i} .

WHY. We can't use $μ$ in a formula because we don't know it. All we have is $\overset{ˉ}{X}$ . The entire mystery of the " $- 1$ " is born from this substitution: we measure spread around the guessed centre, not the true one.

PICTURE. Blue dots on a line, the true centre $μ$ (lavender) and the guessed centre $\overset{ˉ}{X}$ (coral) marked separately — they almost never land on the same spot.

Step 2 — What "spread" should mean: squared distances

WHAT. Spread means "how far are the dots from the centre, typically?" We measure each dot's distance from a centre, square it, and average. The true population version is the variance:

σ^{2} = E [(X_{i} - μ)^{2}] .

WHY square, not absolute value? Squaring makes the algebra split cleanly (Step 5 depends on this), and it matches the variance already defined in Sampling Distributions. It also punishes far-away dots harder, which is what "spread" should feel like.

PICTURE. Each dot connected to a centre by a stick; a shaded square sits on each stick — its area is the squared distance.

Step 3 — The naive estimator, and a suspicion

WHAT. The obvious guess for $σ^{2}$ copies the definition but uses our data and the guessed centre $\overset{ˉ}{X}$ :

\overset{σ}{^}_{naive}^{2} = \frac{1}{n} i = 1 \sum n (X_{i} - \overset{ˉ}{X})^{2} .

WHY suspicious? $\overset{ˉ}{X}$ is built to sit among the dots — it's their balance point. So distances measured from $\overset{ˉ}{X}$ are, on average, smaller than distances from the far-off true $μ$ . The naive average will come out too small. Step 4 turns this hunch into an exact equation.

PICTURE. Same dots measured twice: red sticks to $\overset{ˉ}{X}$ (short) vs lavender sticks to $μ$ (longer). The red squares are visibly smaller.

Step 4 — The key identity: splitting the spread

WHAT. We prove an exact bookkeeping identity — no expectation yet, just algebra true for any fixed dots:

i = 1 \sum n (X_{i} - μ)^{2} = i = 1 \sum n (X_{i} - \overset{ˉ}{X})^{2} + n (\overset{ˉ}{X} - μ)^{2} .

WHY this is the whole game. It says: spread-around-true = spread-around-guess + drift-penalty. Rearranged, spread-around-guess is spread-around-true minus a positive penalty. That's the "too small" from Step 3, now exact.

PICTURE. A balance beam: the true-centre spread (left pan) equals the guessed-centre spread plus a small extra weight labelled $n (\overset{ˉ}{X} - μ)^{2}$ (right pan).

Step 5 — Take the average: two facts we already own

WHAT. Apply $E [\cdot]$ ("on average over all samples") to the identity. Two ingredients, each already established in the parent note and in the LLN circle of ideas:

E [(X_{i} - μ)^{2}] = σ^{2}, E [(\overset{ˉ}{X} - μ)^{2}] = Var (\overset{ˉ}{X}) = \frac{σ ^{2}}{n} .

Now take $E$ of the whole Step-4 identity:

E [i \sum (X_{i} - \overset{ˉ}{X})^{2}] = n σ^{2} i \sum σ^{2} - σ^{2} n \cdot \frac{σ ^{2}}{n} = (n - 1) σ^{2} .

WHY. Expectation is linear (it passes through sums and constants), so nothing here needs new machinery — only the two facts above.

PICTURE. A bar of height $n σ^{2}$ with one $σ^{2}$ -sized chunk (the penalty) sliced off, leaving a bar of height $(n - 1) σ^{2}$ .

Step 6 — Undo the shrink: the $n - 1$ divisor

WHAT. We found the sum of squared deviations averages to $(n - 1) σ^{2}$ . To get something that averages to exactly $σ^{2}$ , divide by $n - 1$ :

s^{2} = \frac{1}{n - 1} i = 1 \sum n (X_{i} - \overset{ˉ}{X})^{2} ⟹ E [s^{2}] = \frac{( n - 1 ) σ ^{2}}{n - 1} = σ^{2} .

WHY $n - 1$ and not some other number? Because Step 5 produced exactly $(n - 1)$ , not $n$ , not $n - 2$ . The divisor is not a fudge — it is the unique number that makes the average come out right. The " $1$ " is the single degree of freedom spent estimating the centre $\overset{ˉ}{X}$ from the same data.

PICTURE. Two side-by-side dials showing $E$ of each estimator: divide-by- $n$ points below the target $σ^{2}$ (biased low), divide-by- $(n - 1)$ points dead on target.

Step 7 — Edge and degenerate cases

WHAT & WHY (each case shown so no scenario surprises you):

Definition All the corners

$n = 1$ (one dot). Then $n - 1 = 0$ and $s^{2} = \frac{0}{0}$ — undefined, and rightly so: with a single point you have no information about spread at all (the one dot is its own mean, distance zero). The formula refusing to answer is honest.
$n = 2$ . Divisor is $1$ : $s^{2} = (X_{1} - \overset{ˉ}{X})^{2} + (X_{2} - \overset{ˉ}{X})^{2} = \frac{1}{2} (X_{1} - X_{2})^{2}$ . Half the squared gap — the smallest sample that can even see spread.
All dots identical. Every $X_{i} - \overset{ˉ}{X} = 0$ , so $s^{2} = 0$ . Correct: zero observed spread.
The penalty is always non-negative. $n (\overset{ˉ}{X} - μ)^{2} \geq 0$ for every sample, so the naive divide-by- $n$ version is always biased low, never high — never zero unless $\overset{ˉ}{X} = μ$ exactly, which has probability $0$ for continuous data.
Large $n$ . The bias of the naive version is $- σ^{2} / n \to 0$ , so both versions become consistent; the $n - 1$ correction only matters for small samples. This is why the parent note says the biased version is still consistent.

PICTURE. A strip of mini-panels: $n = 1$ (a lone dot, no spread possible), $n = 2$ (two dots, one stick), all-equal (dots stacked, spread $= 0$ ), and large- $n$ (bias arrow shrinking toward the axis).

The one-picture summary

Everything above is one sentence with a picture: measuring spread from the guessed centre $\overset{ˉ}{X}$ instead of the true centre $μ$ costs you one $σ^{2}$ , so you average over $n - 1$ effective terms, not $n$ .

Recall Feynman: tell it to a 12-year-old

You want to know how spread-out a bunch of dots are. Real spread means "how far from the true middle." But you don't know the true middle, so you use the dots' own average as a stand-in. Trouble: the dots' own average always sits snugly among the dots, so distances measured from it come out a little too short — every single time. How much too short? We proved it's exactly one dot's worth of spread. So instead of splitting the total among all $n$ dots, we split it among $n - 1$ — pretending one dot "got used up" pinning down the middle. Divide by $n - 1$ and the shortfall is fixed perfectly: on average you get the true spread back. With just one dot there's nothing to divide by ( $n - 1 = 0$ ) — and truly, one dot tells you nothing about spread.

Recall Quick self-test

Why does the middle cross-term vanish in Step 4? ::: Because $\sum_{i} (X_{i} - \overset{ˉ}{X}) = 0$ — the dots balance around their own mean, so that factor is exactly zero. What is the "penalty term" and why is it always positive? ::: $n (\overset{ˉ}{X} - μ)^{2}$ , the squared drift of the guessed centre from the true centre times $n$ ; a square is never negative. Where exactly does the " $- 1$ " come from numerically? ::: From subtracting the penalty $E [n (\overset{ˉ}{X} - μ)^{2}] = σ^{2}$ off $n σ^{2}$ , leaving $(n - 1) σ^{2}$ . Why is the divide-by- $n$ version still consistent? ::: Its bias is $- σ^{2} / n$ , which $\to 0$ as $n \to \infty$ , so it converges to $σ^{2}$ anyway.

Related ideas: Bias–Variance Tradeoff (why an unbiased rule isn't automatically the best), Maximum Likelihood Estimation (which often produces the biased divide-by- $n$ version), Central Limit Theorem and Fisher Information (how tightly data can ever pin $σ^{2}$ down).