Law of Large Numbers — weak and strong: Question bank

This is a rapid-fire self-test on the ideas behind the Law of Large Numbers — no heavy arithmetic, just the traps. Cover the right side, answer out loud, then reveal. Each answer gives you the reasoning, not just a verdict.

Figure — Law of Large Numbers — weak and strong

True or false — justify

True or false: the Strong Law implies the Weak Law.

True. Almost-sure convergence (whole-path settling) is a strictly stronger mode than convergence in probability (per-

n

closeness), so SLLN always gives WLLN — see Modes of Convergence. The reverse fails.

True or false: WLLN implies SLLN.

False. WLLN only pins down each fixed large

n

; the running average may still wander out again infinitely often, which SLLN forbids. In-probability does not force almost-sure.

True or false: the LLN requires the

X_{i}

to have finite variance.

False. Finite

σ^{2}

powers only the easy Chebyshev proof. Khinchin's weak law and Kolmogorov's strong law for i.i.d. variables need only

E ∣ X_{i} ∣ < \infty

(a finite mean).

True or false: the LLN requires the

X_{i}

to be independent.

Largely true for the classical statements — independence makes the covariances vanish so

Var (\overset{ˉ}{X}_{n}) = σ^{2} / n

. There are LLNs for weakly-dependent sequences, but the standard theorem assumes i.i.d.

True or false: the LLN says

\overset{ˉ}{X}_{n}

converges to a Normal (bell) curve.

False — that's the Central Limit Theorem. The LLN says

\overset{ˉ}{X}_{n} \to μ

, a single constant; the CLT describes the leftover wiggle around it.

True or false: if the

X_{i}

have no finite mean, the sample mean still settles down.

False. The Cauchy distribution has no mean, and its sample means never converge — they stay just as spread out as a single draw. Finite mean is the true minimal condition.

True or false: as

n

grows, the sum

\sum_{i = 1}^{n} (X_{i} - μ)

shrinks toward zero.

False. The sum of deviations typically grows (like

n

). It's dividing by

n

that tames the average, not the sum shrinking.

True or false: SLLN guarantees that for a specific infinite coin sequence, the head-fraction reaches exactly

1/2

True for almost every sequence (probability 1). There exist exceptional sequences (e.g. all heads) where it fails, but they form a set of probability zero.

True or false: the Chebyshev bound

σ^{2} / (n ε^{2})

is the tightest possible bound.

False. It is deliberately crude — the CLT gives far sharper tail estimates. Chebyshev is a safe overkill, useful because it needs only the variance

σ^{2}

True or false: "both the weak and strong laws just need a finite mean" is the complete story of the minimal conditions.

False — it is a simplification. It is exact for the i.i.d. case. For merely independent, non-identically distributed

X_{i}

, Kolmogorov's strong law instead requires

\sum_{i} Var (X_{i}) / i^{2} < \infty

. So the minimal condition depends on the setting.

Spot the error

"By Chebyshev,

\sum_{n} P (∣ \overset{ˉ}{X}_{n} - μ ∣ > ε) \leq \sum_{n} \frac{σ ^{2}}{n ε ^{2}}

is finite, so Borel–Cantelli gives SLLN." — where's the error?

The series

\sum 1/ n

diverges, so this bound is not summable. That's exactly why SLLN needs Kolmogorov's sharper maximal inequality, not the naive per-

n

Chebyshev bound — see Borel–Cantelli Lemma.

"Markov's inequality applies to any random variable

Y

." — spot the error.

Markov's Inequality needs

Y \geq 0

(non-negative). Proof sketch in one line:

E [Y] \geq \int_{a}^{\infty} y f (y) d y \geq a P (Y \geq a)

, giving

P (Y \geq a) \leq E [Y] / a

. The first step drops the

[0, a)

mass, which is only harmless when

Y

can't go negative.

"Chebyshev gives

P (∣ X - μ ∣ \geq ε) \leq σ^{2} / ε^{2}

, so with

ε < σ

we get a probability

> 1

— contradiction." — what's wrong?

Nothing is contradicted; the bound is simply vacuous (a probability

\leq

something

> 1

is trivially true). Bounds may exceed 1; that just means they carry no information there.

"Since

Var (\overset{ˉ}{X}_{n}) = σ^{2} / n \to 0

, the average becomes deterministic at finite

n

." — error?

For any finite

n

the variance is positive, so

\overset{ˉ}{X}_{n}

is genuinely random. Only in the limit does the spread vanish; convergence is a statement about

n \to \infty

"After 7 reds at roulette, black is more likely next spin because the LLN forces balance." — error?

Gambler's Fallacy. Spins are independent, so past outcomes don't shift future odds. The LLN balances by dilution — the streak becomes a tiny fraction of a huge

n

— not by any corrective force.

\overset{ˉ}{X}_{n} \to μ

in probability means the sequence

\overset{ˉ}{X}_{n} (ω)

converges for every outcome

ω

." — error?

Convergence in probability is about shrinking probabilities of deviation, not pointwise convergence of paths. Path-by-path convergence is the almost-sure (strong) statement.

Why questions

Why does the variance of

\overset{ˉ}{X}_{n}

shrink like

1/ n

and not

1/ n^{2}

Two effects fight: dividing the sum by

n

scales variance by

1/ n^{2}

, but the sum stacks

n

independent variances (each

σ^{2}

), giving

n σ^{2}

. Multiply:

\frac{1}{n ^{2}} \cdot n σ^{2} = σ^{2} / n

. Independence is what lets the

n

noises partially cancel instead of piling up like

n^{2}

; see the figure below.

Why is the SLLN harder to prove than the WLLN?

WLLN controls one

n

at a time via Chebyshev's Inequality, but SLLN must control the entire tail of the path at once, needing a maximal inequality so the bad-event probabilities become summable for Borel–Cantelli Lemma.

Why does Monte Carlo integration actually work?

An integral

\int_{0}^{1} g (x) d x

equals

E [g (U)]

for uniform

U

, so the SLLN guarantees the sample average of

g (U_{i})

converges almost surely to the integral — see Monte Carlo Methods.

Why does "summable probability of bad events" give almost-sure convergence?

If the total probability of all bad events is finite, only finitely many can occur; after the last one you stay in the good region forever, which is exactly almost-sure convergence.

Why doesn't the LLN contradict the fact that a fair coin can produce a long streak of heads?

Streaks are allowed and even certain to appear; they just get swamped as a fraction of ever-larger

n

. The LLN constrains the ratio, not the raw counts.

Why can't we conclude "half the flips are heads" for any single finite

n

The LLN is a limiting statement. At finite

n

the fraction fluctuates; the CLT says those fluctuations are of order

1/ n

, so exact balance is not required.

Edge cases

What does the LLN say if

X_{i}

is a constant

c

(zero variance)?

The average is

c

for every

n

, so it converges trivially — both laws hold instantly. Zero variance (

σ^{2} = 0

) means the Chebyshev bound is

0

for all

n \geq 1

What happens to the LLN for a Cauchy distribution?

It fails entirely: with no finite mean

μ

\overset{ˉ}{X}_{n}

has the same Cauchy distribution as a single draw, so it never concentrates. This is the canonical counterexample.

What if the

X_{i}

are identically distributed but correlated?

The classical

Var (\overset{ˉ}{X}_{n}) = σ^{2} / n

breaks because covariance terms no longer vanish. Convergence may still hold under weak dependence, but the simple Chebyshev proof no longer applies.

Give a concrete "weakly dependent" case where an LLN still holds.

A stationary sequence whose correlations decay fast enough — e.g. an

m

-dependent sequence (draws more than

m

apart are independent) or a mixing chain where

Cov (X_{1}, X_{k}) \to 0

quickly. Then

Var (\overset{ˉ}{X}_{n})

still

\to 0

(the covariance sum grows slower than

n^{2}

), so

\overset{ˉ}{X}_{n} \to μ

despite mild dependence.

n = 1

, what does the Chebyshev-based WLLN bound say?

It reduces to plain Chebyshev,

P (∣ X_{1} - μ ∣ \geq ε) \leq σ^{2} / ε^{2}

— a valid but often useless single-sample statement, since one draw carries no averaging benefit.

ε \to 0

for fixed

n

, what happens to the WLLN bound?

The bound

σ^{2} / (n ε^{2})

blows up to infinity, i.e. becomes vacuous. You cannot demand arbitrarily tight closeness at a fixed finite

n

; you must send

n \to \infty

too.

Recall One-line summary to lock in

WLLN = "probably close at large $n$ " (in probability, via Chebyshev); SLLN = "eventually close forever" (almost surely, via Borel–Cantelli). Strong ⇒ weak; for i.i.d. variables both need only a finite mean $μ$ (non-i.i.d. Kolmogorov needs $\sum Var (X_{i}) / i^{2} < \infty$ ); neither is the CLT.