Visual walkthrough — Law of Large Numbers — weak and strong

Before we start, three words in plain language (each gets a picture later, but let's name them now):

---

Step 1 — A random variable is mass sitting on a number line

WHAT. Picture the possible outputs of the machine laid out on a horizontal number line. Over each output value we stack a bar whose height is the probability of that value. The total height of all bars is $1$ — the machine always outputs something.

WHY. Every inequality we build is really a statement about how much probability mass sits far from the centre. To reason about "far from the centre" we first need to see mass as a physical thing sitting at positions. This picture turns abstract probability into a stack of weights.

PICTURE. In the figure, the black bars are the probability weights. The red dot marks $\mu$ — literally the balance point where the number line would tip level if the bars were real weights.

$\mu =\mathbb{E}[X]=\underset{\text{add over every possible value}}{\underbrace{\sum _x}}\underset{\text{the position}}{\underbrace{x}}\cdot \underset{\text{the weight there}}{\underbrace{\mathbb{P}(X=x)}}$

Each term is "position times weight" — exactly the recipe for a centre of mass. That is why $\mu$ is the balance point and not just an abstract symbol.

---

Step 2 — Markov's inequality: mass far out costs you mean

WHAT. Take a machine that only outputs values $\ge 0$ (call its output $Y$). Pick a threshold $a>0$. We claim the probability of landing at or past $a$ can't be large if the average $\mathbb{E}[Y]$ is small: $\mathbb{P}(Y\ge a)\le \frac{\mathbb{E}[Y]}{a}.$

WHY. This is the root tool — see Markov's Inequality. It's the only way we know to convert "the average is small" into "not much mass is far out." We need exactly that conversion, because the Weak Law is about mass being far from $\mu$.

PICTURE. Look at the figure. The full black curve is the weight $y\cdot f(y)$ we're summing to get $\mathbb{E}[Y]$ (this is the "position times weight" from Step 1, now for a continuous machine, using a smooth density $f$). We do two shrinking moves, each shaded:

$\mathbb{E}[Y]=\underset{\text{all the mass, weighted by position}}{\underbrace{{\int }_0^{\infty }yf(y)dy}}\ge \underset{\text{threw away }[0,a):\text{ only removed}\ge 0\text{ stuff}}{\underbrace{{\int }_a^{\infty }yf(y)dy}}\ge \underset{\text{on }[a,\infty ),y\ge a}{\underbrace{{\int }_a^{\infty }af(y)dy}}=a\mathbb{P}(Y\ge a)$

• First $\ge$: we deleted the red-shaded left chunk $[0,a)$. Since $Y\ge 0$ everything there was $\ge 0$, so removing it can only lower the total. What it looks like: chop off the left, the pile gets no taller.
• Second $\ge$: on the right chunk every $y$ is at least $a$, so replacing each $y$ by the smaller constant $a$ again lowers the total. What it looks like: flatten the right hump down to a rectangle of height $a$.
• The final rectangle's area is $a$ times its total probability $\mathbb{P}(Y\ge a)$.

Divide both sides by $a$ and you have Markov. The integral $\int$ here is just "add up infinitely many thin weights" — the continuous version of the sum in Step 1. That's why the integral appears and not something fancier.

---

Step 3 — Chebyshev: turn Markov into a two-sided ruler around $\mu$

WHAT. Markov only talks about non-negative $Y$. But we care about $X$ landing far from $\mu$ on either side. Trick: feed Markov the machine $Y=(X-\mu {)}^2$, which is always $\ge 0$, and choose the threshold $a={\epsilon }^2$.

WHY. Squaring does two jobs at once: it makes the quantity non-negative (so Markov applies) and it treats "too far left" and "too far right" identically, because $(X-\mu {)}^2$ ignores sign. That is exactly the two-sided statement we want. See Chebyshev's Inequality.

PICTURE. The figure shows the parabola $y=(x-\mu {)}^2$. The two vertical red lines at $\mu -\epsilon$ and $\mu +\epsilon$ are where $(x-\mu {)}^2={\epsilon }^2$. Being outside the red band on the number line is the same event as the parabola rising above the height ${\epsilon }^2$.

$\underset{\text{parabola above the red line}}{\underbrace{(X-\mu {)}^2\ge {\epsilon }^2}}⟺\underset{\text{outside the red band}}{\underbrace{|X-\mu |\ge \epsilon }}$

Now apply Markov to $Y=(X-\mu {)}^2$ with $a={\epsilon }^2$: $\mathbb{P}(|X-\mu |\ge \epsilon )=\mathbb{P}((X-\mu {)}^2\ge {\epsilon }^2)\le \frac{\mathbb{E}[(X-\mu {)}^2]}{{\epsilon }^2}=\frac{{\sigma }^2}{{\epsilon }^2}$

• $\mathbb{E}[(X-\mu {)}^2]$ is by definition the variance ${\sigma }^2$ from Step 0 — the average squared spread.
• $\epsilon$ is our chosen tolerance ("how far is too far").
• Reading: wide spread $\Rightarrow$ more mass allowed outside the band; big tolerance $\Rightarrow$ less.

---

Step 4 — The averaging machine and why its spread shrinks

WHAT. Now build the sample mean machine. Press the button $n$ times, get $X_1,\dots ,X_n$, and output their average ${\bar{X}}_n=\frac{1}{n}{\sum }_{i=1}^n{X}_i.$ We compute its mean and — the crucial part — its variance.

WHY. Chebyshev (Step 3) needs the variance of whatever machine we point it at. If we can show ${\bar{X}}_n$ has a tiny variance, Chebyshev will immediately force its mass to hug $\mu$. So this step is where the shrinking actually happens.

PICTURE. The figure stacks the spread of a single $X$ (wide black bell) against the spread of ${\bar{X}}_n$ for growing $n$ (progressively narrower curves, the narrowest drawn in red). Same centre $\mu$, but the averaging machine's spread collapses inward.

Mean (uses linearity — average of averages is the average): $\mathbb{E}[{\bar{X}}_n]=\frac{1}{n}{\sum }_{i=1}^n\underset{=\mu \text{ each}}{\underbrace{\mathbb{E}[X_i]}}=\frac{1}{n}\cdot n\mu =\mu$ The centre never moves. Good — the machine is aimed correctly.

Variance (uses independence — this is where "i.i.d." earns its keep): $\mathrm{Var}({\bar{X}}_n)=\mathrm{Var}(\frac{1}{n}\sum X_i)=\underset{\text{constant }\frac{1}{n}\text{ pulls out squared}}{\underbrace{\frac{1}{n^2}}}\underset{\text{independence: cross terms vanish}}{\underbrace{\sum _{i=1}^n\mathrm{Var}(X_i)}}=\frac{n{\sigma }^2}{n^2}=\frac{{\sigma }^2}{n}$

• Pulling $\frac{1}{n}$ out of a variance squares it to $\frac{1}{n^2}$ (variance scales with the square of a multiplier).
• Because the $X_i$ are independent, the variance of the sum is just the sum of the variances — no covariance cross-terms survive. That is the single place independence is used.
• Result: $\mathrm{Var}({\bar{X}}_n)={\sigma }^2/n$. This $1/n$ is the whole engine. Every extra draw divides the spread again.

---

Step 5 — Assemble the Weak Law

WHAT. Point Chebyshev (Step 3) at the averaging machine (Step 4), whose mean is $\mu$ and variance is ${\sigma }^2/n$.

WHY. All the parts now click: Chebyshev gives a bound in terms of variance, and Step 4 made that variance small. Substituting is the finish line.

PICTURE. The figure shows the bound $\frac{{\sigma }^2}{n{\epsilon }^2}$ as a black curve falling toward $0$ as $n$ grows, with the shaded "probability of being off by more than $\epsilon$" (red) squeezed underneath it — trapped, forced to zero.

$\mathbb{P}(|{\bar{X}}_n-\mu |\ge \epsilon )\le \frac{\mathrm{Var}({\bar{X}}_n)}{{\epsilon }^2}=\frac{{\sigma }^2/n}{{\epsilon }^2}=\underset{\to 0\text{ as }n\to \infty }{\underbrace{\frac{{\sigma }^2}{n{\epsilon }^2}}}$

• Numerator ${\sigma }^2$: fixed spread of one draw.
• Denominator $n{\epsilon }^2$: grows without bound as $n\to \infty$ (for any fixed tolerance $\epsilon$).
• A fixed number over an exploding number $\to 0$.

---

Step 6 — The degenerate & edge cases (never leave a gap)

WHAT / WHY / PICTURE for each corner the reader could hit:

---

The one-picture summary

WHAT. One figure that chains the whole argument: a single draw's wide spread → the parabola trick that turns "far from $\mu$" into "square exceeds ${\epsilon }^2$" → the $1/n$ variance collapse → the bound crushing the tail probability to zero.

$\underset{\text{Markov}}{\underbrace{Y\ge 0}}\stackrel{Y=(X-\mu {)}^2}{\to }\underset{\text{Chebyshev}}{\underbrace{\mathbb{P}(|X-\mu |\ge \epsilon )\le \frac{{\sigma }^2}{{\epsilon }^2}}}\stackrel{X\to {\bar{X}}_n,\mathrm{Var}=\frac{{\sigma }^2}{n}}{\to }\underset{\text{Weak Law}}{\underbrace{\mathbb{P}(|{\bar{X}}_n-\mu |\ge \epsilon )\le \frac{{\sigma }^2}{n{\epsilon }^2}\to 0}}$

Recall Feynman retelling — the whole walkthrough in plain words

Picture probability as little weights sitting on a number line; the mean $\mu$ is the point where they'd balance. Markov's idea: if the average position is small, you can't have much weight parked way out far — you'd tip the balance. To use that on both sides of $\mu$, we square the distance $(X-\mu {)}^2$ (squaring forgets the sign and can't go negative), feed it to Markov, and out pops Chebyshev: the chance of being more than $\epsilon$ from $\mu$ is at most ${\sigma }^2/{\epsilon }^2$. Now the magic move: instead of one draw, average $n$ independent draws. Averaging keeps the balance point at $\mu$ but — because the independent wiggles cancel — squeezes the spread down to ${\sigma }^2/n$. Feed that shrunken spread back into Chebyshev and the bound becomes ${\sigma }^2/(n{\epsilon }^2)$, which slides to zero as $n$ grows. So the average is probably close to $\mu$, and the more you average the surer you are. The only things that can break this: no balance point exists (Cauchy — no mean), or the draws secretly lean on each other (dependence stops the wiggles from cancelling).

Recall One-line challenge before you leave

Why does averaging shrink the spread but not move the centre? ::: Linearity keeps $\mathbb{E}[{\bar{X}}_n]=\mu$ (centre fixed); independence makes variances add, and the $1/n^2$ from pulling out $\frac{1}{n}$ beats the $n$ terms, leaving ${\sigma }^2/n$ (spread shrinks).