Visual walkthrough — Expectation, variance, and standard deviation

This page builds, from nothing, the rule that measures how spread out a random variable is — and its famous shortcut. We will not use a single symbol before it has a picture attached: we first meet the averaging operator, then the spread measure, then its shortcut, then its square-root cousin. By the end you will see why we square, why the shortcut is true, and what happens in the strange cases (a constant, a rescaled variable, two variables added together).

Parent: Expectation, variance, and standard deviation. If the words "random variable" or "probability" feel new, read random variables & distributions first.

---

Step 1 — A random variable is dots on a number line

WHAT. A random variable $X$ is just a rule that spits out a number when you run an experiment. Roll a die → you get one of $1,2,3,4,5,6$. Each possible number $x_i$ has a weight $p_i=P(X=x_i)$: how often it shows up in the long run. All weights are between $0$ and $1$ and they add to $1$.

WHY. Before we can talk about "spread," we need something to spread. Picture the possible values as dots sitting on a horizontal number line, and let the size of each dot be its probability. Big dot = happens often.

PICTURE. Six equal dots for a fair die — each carries weight $\frac{1}{6}$.

---

Step 2 — Expectation is the balance point

WHAT. The expectation $\mu =E[X]$ is the balance point of those weighted dots: $\mu =\underset{\text{add over all values}}{\underbrace{\sum _i}}\underset{\text{position}}{\underbrace{x_i}}\cdot \underset{\text{its weight}}{\underbrace{p_i}}$

WHY. Imagine the number line is a see-saw and each dot is a physical weight sitting at position $x_i$. The point where the see-saw balances is exactly ${\sum }_i{x}_i{p}_i$. That is the single number that best represents "where the values sit." For a fair die it lands at $3.5$ — between $3$ and $4$, because the weights are symmetric.

PICTURE. A triangular pivot ▲ slid under the dots until they balance, at $\mu =3.5$.

---

Step 3 — Deviation: how far each dot sits from the pivot

WHAT. For each value define its deviation $d_i=x_i-\mu .$ This is a signed arrow from the pivot to the dot: positive (pointing right) if $x_i>\mu$, negative (pointing left) if $x_i<\mu$, zero if the dot sits on the pivot.

WHY. "Spread" should mean "how far are the dots from the center, typically." So the raw material we need is the collection of distances $d_i$. But there is a trap: if we just average the signed deviations we get $E[X-\mu ]=E[X]-\mu =\mu -\mu =0.$ Every distribution! The left arrows exactly cancel the right arrows — that is literally what "balance point" meant in Step 2. So the naive average distance is always zero and tells us nothing. We must kill the signs.

PICTURE. Blue arrows pointing left (negative $d_i$), pink arrows pointing right (positive $d_i$), summing to zero.

---

Step 4 — Square each arrow to remove the sign

WHAT. Replace each signed arrow $d_i$ by its square $d_i^2=(x_i-\mu {)}^2$. Geometrically, an arrow of length $d_i$ becomes a square with side $|d_i|$ and area $d_i^2$. Area is never negative, so both left and right arrows now contribute positively.

WHY squaring, not absolute value? Two reasons, both visible in the picture.

1. Squares can't cancel. A left arrow and a right arrow of the same length give the same area, so they add up instead of cancelling.
2. Smooth algebra. $|d|$ has a sharp kink at $0$ (no clean slope there), which wrecks calculus. $d^2$ is a smooth bowl — you can differentiate it everywhere. That smoothness is exactly what gradient descent and MSE loss rely on.

The absolute-value version exists (Mean Absolute Deviation) but loses these two gifts.

PICTURE. Each arrow grows into a shaded square; long arrows become much bigger squares (area grows like length²), so far-off values are emphasized.

---

Step 5 — Variance is the average of the squares

WHAT. The variance is the probability-weighted average of those square areas: $\text{Var}(X)=\underset{\text{over all values}}{\underbrace{\sum _i}}\underset{\text{square area}}{\underbrace{(x_i-\mu {)}^2}}\cdot \underset{\text{its weight}}{\underbrace{p_i}}=E[(X-\mu {)}^2].$

WHY. Big squares that happen often (big area × big weight) dominate; rare far-out values contribute area but only a little weight. So variance is the typical square-area — one number summarizing the spread. It is the balance point of the squares, just as $\mu$ was the balance point of the values.

PICTURE. The see-saw again, but now the weights are the square areas; the balance point of those areas is $\text{Var}(X)$.

---

Step 6 — The shortcut: $\text{Var}(X)=E[X^2]-{\mu }^2$

WHAT. Expand the square inside the expectation. Watch each term:

$$(X-\mu {)}^2=\underset{\text{square of value}}{\underbrace{X^2}}-\underset{\text{cross term}}{\underbrace{2\mu X}}+\underset{\text{constant}}{\underbrace{{\mu }^2}}.$$

Take expectation of each piece. Expectation is linear (it passes through $+$ and pulls out constants), and $\mu$ is just a fixed number:

$$E[(X-\mu {)}^2]=\underset{\text{avg of squares}}{\underbrace{E[X^2]}}-\underset{=2\mu \cdot \mu }{\underbrace{2\mu E[X]}}+\underset{\text{constant}\Rightarrow E[{\mu }^2]={\mu }^2}{\underbrace{{\mu }^2}}=E[X^2]-2{\mu }^2+{\mu }^2.$$

Combine the last two: $\boxed{\text{Var}(X)=E[X^2]-{\mu }^2=E[X^2]-(E[X]{)}^2}$

WHY this form? To compute it you only need two easy averages — the average of $x_i$ (gives $\mu$) and the average of $x_i^2$ — then subtract ${\mu }^2$. No need to compute every deviation. Note $E[X^2]\ge (E[X]{)}^2$ always, so this is guaranteed $\ge 0$: that inequality is Step 5's non-negativity in disguise.

PICTURE. Left tower = $E[X^2]$ (average height of the squared values). We chop off a block of height ${\mu }^2$; what remains is the variance.

---

Step 7 — Edge cases: constant, rescale, and shift

WHAT & WHY. A formula you don't test at the extremes is a formula you don't understand. Three degenerate inputs:

• A constant $c$ (all weight on one dot). No spread: every square area is $0$. $\text{Var}(c)=E[(c-c{)}^2]=0.$
• Rescale $aX$ (stretch the line by factor $a$). Every arrow length scales by $|a|$, so every area scales by $a^2$: $\text{Var}(aX)=a^2\text{Var}(X).$ For the standard deviation this becomes $\text{SD}(aX)=|a|\text{SD}(X)$ — the absolute value appears because the square root of $a^2$ is $|a|$, not $a$. (This matters for feature scaling: dividing by $\sigma$ makes the new SD exactly $1$.)
• Shift $X+b$ (slide every dot right by $b$). The pivot slides by $b$ too, so distances don't change: $\text{Var}(X+b)=\text{Var}(X).$

PICTURE. Three mini-boards: a lone dot (variance $0$), a stretched set of squares ($a^2$ bigger), a slid set (identical squares).

---

The one-picture summary

Values on a line → find the balance point $\mu$ → measure signed arrows $d_i=x_i-\mu$ (they cancel!) → square them into areas → average the areas = variance → algebra rewrites it as $E[X^2]-{\mu }^2$ → square-root gives SD in original units.

Recall Feynman retelling — say it like a story

Picture beads on a wire, each bead heavier if it happens more often. Slide a pivot under the wire until it balances — that spot is the mean. Now I want to know "how scattered are the beads?" My first idea: measure how far each bead is from the pivot and average those distances. But that's always zero — the beads on the left pull the average down exactly as much as the beads on the right pull it up; it's the balance point, after all. So I cheat the signs away by squaring each distance, turning every arrow into a little square whose area can't be negative. I average those square-areas, weighting by how often each bead shows up — that average is the variance. Squaring also punishes far-away beads extra hard, and it keeps the math smooth so calculus works. Finally, since I squared the distances, the units got squared too (metres became metres²), so I take a square root at the end to get the standard deviation, back in honest units. A clever algebra trick, expanding $(x-\mu {)}^2$, shows the whole thing equals "average of the squares minus the square of the average," which is the fast way to compute it. Constants have zero variance (no scatter), stretching the wire by $a$ multiplies variance by $a^2$, and sliding it sideways changes nothing.

Recall Quick self-test

Why is the average signed deviation always zero? ::: Because $\mu$ is the balance point, so left and right deviations cancel: $E[X-\mu ]=0$. Why square instead of absolute value? ::: Squares can't cancel by sign and are smooth (differentiable), giving clean algebra and calculus. State the shortcut formula. ::: $\text{Var}(X)=E[X^2]-(E[X]{)}^2$. What is $\text{Var}(aX+b)$? ::: $a^2\text{Var}(X)$ — the shift $b$ does nothing, the scale $a$ enters squared. Why take the square root for SD? ::: To undo the squaring and return to the original data units.