Foundations — Law of large numbers

Before we can even read the parent note's formulas, we must earn every piece of notation. We'll go in order: from a single random draw, to a list of draws, to their average, to the two Greek letters that describe where that average is heading and how wildly it wobbles. Each new symbol gets a plain meaning, a picture, and a reason the topic can't live without it.

---

1. A random variable — the symbol $X$

Plain words: flip a coin — the outcome is "heads" or "tails," which are words, not numbers. A random variable is the translator that says "heads $\to 1$, tails $\to 0$." Now every flip produces a number we can average.

The picture: think of a machine with a slot. You feed in a random event (a coin toss), and a number pops out the bottom.

Why the topic needs it: the Law of Large Numbers is entirely about averaging numbers. You cannot average the word "heads." $X$ is the bridge from "stuff happening" to "arithmetic we can do."

Recall

What does a capital $X$ stand for here? ::: A random variable — a rule assigning a number to each random outcome.

---

2. Many draws — subscripts $X_1,X_2,\dots ,X_n$

Plain words: the subscript is just a name tag. $X_5$ is not "$X$ times 5" — it is "the fifth draw." The $i$ is a placeholder that can be any tag from $1$ up to $n$.

The picture: a row of buckets, each holding one draw's number, labelled left to right.

Why the topic needs it: the whole story is "what happens as $n$ grows." We need a way to talk about draw number $1$, draw number $1,000,000$, and everything between — subscripts give us that vocabulary.

---

3. The summation sign — ${\sum }_{i=1}^n{X}_i$

Plain words: it is a compact way to write a long "$+++$" chain without knowing exactly how many terms there are.

Why this tool and not just "$+$"? Because $n$ can be huge or unknown. We literally cannot write out a million plus-signs, and we want a formula that works for any $n$. Sigma is the tool that says "add all of them" in one clean symbol.

Recall

In ${\sum }_{i=1}^n$, what do the bottom "1" and top "$n$" tell you? ::: Where to start counting (draw 1) and where to stop (draw $n$).

---

4. The sample mean — ${\bar{X}}_n$

Plain words: it is the ordinary "add-em-up-and-divide" average you already know, just written with our new symbols. Total heads divided by total flips = proportion of heads.

The picture: pile all the bucket-numbers onto one seesaw. The balance point of that seesaw is ${\bar{X}}_n$.

Why the topic needs it: ${\bar{X}}_n$ is the star of the entire show. The Law of Large Numbers is a statement about what this one quantity does as $n$ climbs. Everything else exists to describe where ${\bar{X}}_n$ goes.

---

5. Expected value — the symbol $\mu =E[X]$

Plain words: for a fair coin, half the flips are heads ($1$) and half tails ($0$), so the forever-average is $0.5$. That $0.5$ is $\mu$. Note the difference from ${\bar{X}}_n$: the sample mean ${\bar{X}}_n$ is what you actually measured from $n$ real flips (it wobbles); $\mu$ is the ideal target it wobbles around (it never moves).

The picture: on the number line, $\mu$ is a fixed flag stuck in the ground. ${\bar{X}}_n$ is a dot that dances near the flag and, as $n$ grows, dances closer and closer.

Why the topic needs it: the Law of Large Numbers is precisely "${\bar{X}}_n$ heads toward $\mu$." Without a symbol for the target, we have nothing to converge to. See Expected Value for the full construction of $E[X]$.

Recall

Which one moves as you collect data, $\mu$ or ${\bar{X}}_n$? ::: ${\bar{X}}_n$ moves (it's measured); $\mu$ is a fixed target.

---

6. Variance and ${\sigma }^2$ — how wide the wobble is

Plain words: two experiments can share the same target $\mu$ yet feel totally different — one calm, one chaotic. ${\sigma }^2$ is the number that captures that chaos. It's the average of the squared distances from the mean (squaring makes every distance positive and punishes big misses harder).

Why the topic needs it: the parent's key formula is $\text{Var}({\bar{X}}_n)={\sigma }^2/n$. That single fact — the wobble of the average shrinks as $n$ grows — is the engine of the whole proof. You cannot understand "why more data helps" without ${\sigma }^2$. Full treatment in Variance and Standard Deviation.

Recall

What does a small ${\sigma }^2$ tell you about the draws? ::: They cluster tightly around the mean $\mu$.

---

7. Probability of an event — $P(\cdot )$

The parent writes things like $P(|{\bar{X}}_n-\mu |\ge ϵ)$. Read it inside-out:

• ${\bar{X}}_n-\mu$ = how far your measured average sits from the true target.
• $|\cdots |$ = absolute value: strip the sign, keep only the distance (missing by $-0.3$ and $+0.3$ are equally "off").
• $\ge ϵ$ = "that distance is at least $ϵ$" (a small threshold — the Greek letter epsilon, meaning "a tiny amount you pick").
• $P(\dots )$ = the chance that all of the above is true.

So the whole expression means: "the probability that the sample mean misses the target by more than a whisker $ϵ$." The Law says this probability crashes to $0$ as $n$ grows.

Recall

Why the absolute-value bars in $|{\bar{X}}_n-\mu |$? ::: We care about distance from the target, not direction — over and under both count as "off."

---

8. The limit — ${\lim }_{n\to \infty }$

Plain words: the fraction ${\sigma }^2/(n{ϵ}^2)$ gets smaller every time $n$ grows. Plug in $n=10$, then $100$, then $1,000,000$: the value marches toward $0$ but never is a fixed stopping point — it just heads there. The limit is the destination of that march.

Why the topic needs it: "more and more repetitions" is $n\to \infty$. The Law of Large Numbers is a statement about a limit, so this symbol is unavoidable.

---

9. i.i.d. — the fine print on the draws

Why the topic needs it: independence is exactly what lets variances add in Step 3 of the parent's proof ($\text{Var}$ of a sum = sum of $\text{Var}$s). "Identical" is what lets us write one shared $\mu$ and ${\sigma }^2$ for all draws. Break either and the proof collapses. Deep dive: Independent and Identically Distributed.

---

How these feed the theorem

Read top to bottom: numbers ($X$) become a list, the list becomes an average (${\bar{X}}_n$), we measure its distance from the target ($\mu$), variance tells us the wobble shrinks, and the limit turns "shrinks" into "vanishes" — which is exactly the Law of Large Numbers.

---

Where these tools reappear

Once you own these symbols, they unlock the neighbours: Chebyshev Inequality uses ${\sigma }^2$, $ϵ$, and $P(\cdot )$ to bound the miss-probability; the Central Limit Theorem refines "${\bar{X}}_n$ near $\mu$" into an exact bell shape; Monte Carlo Methods and Bootstrap Sampling turn the averaging idea into estimation algorithms; and Stochastic Gradient Descent leans on the same "mini-batch average $\to$ true average" logic.

---

Equipment checklist

Test yourself — cover the right side and answer before revealing.

• A capital $X$ means... ::: a random variable: a rule turning a random outcome into a number.
• The subscript in $X_i$ means... ::: which draw (a name tag), not multiplication.
• ${\sum }_{i=1}^n{X}_i$ expands to... ::: $X_1+X_2+\cdots +X_n$, the sum of all $n$ draws.
• ${\bar{X}}_n$ is computed by... ::: adding all $n$ draws and dividing by $n$ — the sample average.
• $\mu =E[X]$ is... ::: the fixed long-run (forever) average, the target that ${\bar{X}}_n$ chases.
• The difference between $\mu$ and ${\bar{X}}_n$ is... ::: $\mu$ is the fixed ideal; ${\bar{X}}_n$ is the wobbling measured value.
• ${\sigma }^2$ measures... ::: variance — how widely the draws spread around $\mu$.
• $|{\bar{X}}_n-\mu |\ge ϵ$ describes... ::: the average missing the target by at least a small threshold $ϵ$.
• $P(\text{event})$ is a number in the range... ::: $0$ (impossible) to $1$ (certain).
• ${\lim }_{n\to \infty }$ asks... ::: what value the expression approaches as $n$ grows without bound.
• "i.i.d." requires... ::: draws that are independent and share the same distribution ($\mu$, ${\sigma }^2$).