Visual walkthrough — Expected value, variance, standard deviation — properties

---

Step 0 — The two words we must earn first

Before any formula, we need two ideas anchored to a picture.

Reading the figure. Each coral blob is a probability lump sitting at its value. The lavender triangle is the knife edge — it sits exactly under $\mu$. Term by term:

• $x_i$ — where the lump sits (position on the ruler),
• $p_i$ — how heavy the lump is (its probability share),
• $x_i{p}_i$ — the "turning effort" (torque) that lump exerts about the origin,
• ${\sum }_i{x}_i{p}_i$ — total turning effort, which the balance point $\mu$ must cancel.

---

Step 1 — WHAT does "spread" even mean?

WHAT. We want ONE number saying how far the lumps sit from the balance point, on average.

WHY. Two experiments can share the exact same mean $\mu$ but feel totally different: one tight cluster, one wild scatter. The mean alone is blind to this. We need a spread number.

PICTURE. Look at each lump's signed distance $d_i=x_i-\mu$ — an arrow from the knife edge out to the lump.

• $d_i=x_i-\mu$ — the arrow from the balance point to lump $i$. Mint = the lump is to the right ($d_i>0$); coral = to the left ($d_i<0$).

The tempting first idea: just average these arrows, $\mathbb{E}[d]=\mathbb{E}[X-\mu ]$. Next step shows why that fails.

---

Step 2 — Why raw distances are useless (they sum to zero)

WHAT. Compute the average of the signed arrows $d_i=x_i-\mu$.

WHY. To show it always collapses to $0$ — so we must fix it.

$\mathbb{E}[X-\mu ]=\underset{=\mu }{\underbrace{\mathbb{E}[X]}}-\underset{\text{a constant}}{\underbrace{\mu }}=\mu -\mu =0.$

Term by term: $\mathbb{E}[X]$ is the balance point $\mu$ itself; $\mu$ is a fixed number so its average is just $\mu$. They cancel. This is not a coincidence — the mean is defined to be the point where left-pull and right-pull exactly balance.

PICTURE. The mint (right) arrows and coral (left) arrows have equal total "length-times-weight". They annihilate.

---

Step 3 — The fix: square each arrow

WHAT. Replace each signed distance $d_i$ by its square $d_i^2=(x_i-\mu {)}^2$, then average.

WHY squaring and not, say, absolute value $|d_i|$? Three reasons, all real:

1. Squaring kills the sign (a negative squared is positive) — so nothing cancels.
2. It punishes big misses more — a lump twice as far contributes four times the spread.
3. It is smooth (a parabola), so later algebra and calculus behave; $|d|$ has a sharp corner.

This defines variance: $\mathrm{Var}(X)=\mathbb{E}[(X-\mu {)}^2].$

• $(X-\mu )$ — the signed arrow,
• $(\cdot {)}^2$ — the sign-remover that also magnifies large arrows,
• $\mathbb{E}[\cdots ]$ — average over all lumps, weighting by $p_i$.

PICTURE. Each arrow $d_i$ becomes the area of a square built on it. Left and right both give genuine (positive) area.

Recall Why not just

$\mathbb{E}[|X-\mu |]$? It is a valid spread measure (the "mean absolute deviation"), but its corner at $0$ makes it non-differentiable and its algebra ugly. Squaring gives the clean expandable formula we chase next. ::: Because $x^2$ is smooth and expands algebraically, while $|x|$ has a non-differentiable corner.

---

Step 4 — Expand the square (the key algebraic move)

WHAT. Multiply out $(X-\mu {)}^2$ inside the expectation.

WHY. The definition $\mathbb{E}[(X-\mu {)}^2]$ needs $\mu$ known before you compute — a two-pass job. Expanding lets us reorganise into pieces we can compute in one pass through the data.

$$(X-\mu {)}^2=X^2-2\mu X+{\mu }^2.$$

• $X^2$ — the value squared (varies with the outcome),
• $-2\mu X$ — a cross term; note $\mu$ is a fixed constant here,
• ${\mu }^2$ — a pure constant (the balance point squared).

PICTURE. The classic square-of-a-sum: a big square of side $|X-\mu |$ decomposes into an $X^2$ block, a ${\mu }^2$ block, and two identical $\mu X$ rectangles.

---

Step 5 — Average each piece using linearity

WHAT. Take $\mathbb{E}[\cdots ]$ of the expanded expression, piece by piece.

WHY linearity is legal here: averaging a sum is the sum of the averages, and a constant slides out of an average. (That's the linearity rule the parent proved.)

$$\mathrm{Var}(X)=\mathbb{E}[X^2]-2\mu \mathbb{E}[X]+{\mu }^2.$$

• $\mathbb{E}[X^2]$ — average of the squares (needs LOTUS: average $g(x)=x^2$),
• $-2\mu \mathbb{E}[X]$ — the constant $2\mu$ pulled out front of the average,
• ${\mu }^2$ — average of a constant is the constant itself.

Now substitute the fact $\mathbb{E}[X]=\mu$:

$$\mathrm{Var}(X)=\mathbb{E}[X^2]-2\mu \cdot \mu +{\mu }^2=\mathbb{E}[X^2]-2{\mu }^2+{\mu }^2=\mathbb{E}[X^2]-{\mu }^2.$$

The two middle terms $-2{\mu }^2+{\mu }^2$ collapse to $-{\mu }^2$. Writing $\mu =\mathbb{E}[X]$ back out:

PICTURE. Two competing bars: the tall $\mathbb{E}[X^2]$ bar, minus the shorter $(\mathbb{E}[X]{)}^2$ bar; the leftover slab is the variance.

---

Step 6 — Why the answer is never negative

WHAT. Confirm $\mathrm{Var}(X)\ge 0$ always — i.e. $\mathbb{E}[X^2]\ge (\mathbb{E}[X]{)}^2$.

WHY it must be so. Variance was defined in Step 3 as an average of squares $(X-\mu {)}^2$. Squares are never negative, and averaging non-negative things stays non-negative. So the tall bar can never be shorter than the short bar.

\quad\Longrightarrow\quad \mathbb{E}[X^2]\ge(\mathbb{E}[X])^2.$$ This is exactly [[Jensen's Inequality|Jensen's inequality]] for the convex function $x^2$: *the average of the squares beats the square of the average.* **PICTURE.** A convex parabola $y=x^2$: the chord (which represents $\mathbb{E}[X^2]$) sits **above** the curve point at the mean (which is $(\mathbb{E}[X])^2$). The gap between them *is* the variance. --- ## Step 7 — Edge cases (nothing left unshown) **Case A — a certain outcome (no randomness).** If $X$ always equals one value $c$ (so $p=1$ there), every lump sits on the balance point, so every arrow $d_i=0$. Variance $=\mathbb{E}[0]=0$. The tall and short bars are equal. *A sure thing has zero spread.* **Case B — shift by $b$.** Slide every value: $X\to X+b$. Every lump and the knife edge move together, so all arrows $d_i$ are **unchanged**. Hence $\operatorname{Var}(X+b)=\operatorname{Var}(X)$. The $b$ vanishes. **Case C — scale by $a$.** Stretch: $X\to aX$. Every arrow becomes $a$ times longer, so its square becomes $a^2$ times bigger: $\operatorname{Var}(aX)=a^2\operatorname{Var}(X)$. The factor is **squared** because variance lives in squared units — and it's $a^2\ge 0$ even for negative $a$ (a mirror-flip doesn't shrink spread). - Left panel: certain outcome — one lump, zero arrows, $\operatorname{Var}=0$. - Middle panel: shift — picture slides, arrows identical, $\operatorname{Var}$ identical. - Right panel: scale — picture stretches, arrows $\times a$, variance $\times a^2$. > [!example] Sanity check — the fair die > $\mathbb{E}[X]=3.5$, $\ \mathbb{E}[X^2]=\dfrac{91}{6}\approx 15.1\overline{6}$. > $$\operatorname{Var}(X)=\tfrac{91}{6}-(3.5)^2=15.1\overline{6}-12.25=2.91\overline{6}.$$ > Positive ✓ (Step 6). And $Y=10X+5$: shift does nothing, scale squares → $\operatorname{Var}(Y)=10^2\cdot 2.91\overline{6}=291.\overline{6}$ (Step 7 C). $\ \operatorname{SD}(Y)=|10|\cdot\sqrt{2.91\overline{6}}\approx 17.08$. --- ## The one-picture summary Everything on one canvas: lumps on a ruler → arrows from the balance point → squares of those arrows → their average is variance → which equals the tall bar $\mathbb{E}[X^2]$ minus the short bar $(\mathbb{E}[X])^2$ → and that gap can never be negative. > [!recall]- Feynman retelling (say it out loud) > Scatter some clay lumps on a ruler; the weights are the probabilities. Find where it balances — that's the **mean**. Now I want to know how spread out the clay is. I draw an arrow from the balance point to each lump. If I just average those arrows I get zero, because the left-pull and right-pull cancel — that's what "balance point" means! So instead I build a little square on each arrow (squaring wipes out the sign and punishes far-flung lumps extra), and I average those squares. That average is the **variance**. With a bit of algebra, that same average splits neatly into "the average of the squared values" minus "the square of the average value" — the tall bar minus the short bar. Because I started from averaging squares, the answer can never dip below zero: the tall bar always wins. Slide the whole ruler and nothing changes; stretch it by a factor and the spread grows by that factor *squared*. > [!mnemonic] Remember the formula > **"Mean of the square, minus the square of the mean."** Order matters — swap them and you'd get a negative number, which spread can never be. --- **Prerequisites & neighbours:** [[Probability Distributions]] · [[Covariance and Correlation]] · [[Jensen's Inequality]] · [[Bernoulli and Binomial Distributions]] · [[Law of Large Numbers]] · [[Central Limit Theorem]] · [[Moment Generating Functions]]