2.7.5Statistics & Probability — Intermediate

Probability — classical, empirical, axiomatic (Kolmogorov axioms)

1,761 words8 min readdifficulty · medium

WHAT are we measuring?

WHY sets? Because "and", "or", "not" become intersection \cap, union \cup, complement AcA^c. Logic ↔ set algebra, so we can compute with events.


1. Classical probability

WHY it works: if every outcome is equally likely, each has probability 1n\tfrac1n (they must sum to 1). Event AA is the union of its mm outcomes, so P(A)=m1nP(A)=m\cdot\tfrac1n.

HOW / limitation: needs a symmetry argument to justify "equally likely". Fails for a bent coin, or for "will it rain?" (no symmetric outcomes to count).


2. Empirical (frequentist) probability

WHY the limit: short runs are noisy (10 tosses can give 7 heads). The Law of Large Numbers says the relative frequency settles down toward a fixed number as NN grows — that stable value is the probability.


3. Axiomatic probability (Kolmogorov, 1933)

WHY needed: classical assumes symmetry; empirical needs infinite trials. Kolmogorov asked instead: what minimal rules must any "probability" satisfy? Then classical & empirical both become valid ways to assign numbers inside this framework.

Everything else is derived from these — nothing extra assumed.

Derivations from scratch

Figure — Probability — classical, empirical, axiomatic (Kolmogorov axioms)

Worked examples tying it together


Common mistakes (steel-manned)


Flashcards

Classical probability formula and its key assumption
P(A)=m/nP(A)=m/n; requires all nn outcomes equally likely.
Empirical probability definition
P(A)=limNf/NP(A)=\lim_{N\to\infty} f/N, the relative frequency over many trials.
State the three Kolmogorov axioms
(1) P(A)0P(A)\ge0; (2) P(S)=1P(S)=1; (3) disjoint events add: P(Ai)=P(Ai)P(\cup A_i)=\sum P(A_i).
Prove P()=0P(\varnothing)=0
P(S)=P(S)=P(S)+P()P()=0P(S)=P(S\cup\varnothing)=P(S)+P(\varnothing)\Rightarrow P(\varnothing)=0.
Complement rule and its proof
P(Ac)=1P(A)P(A^c)=1-P(A); from A,AcA,A^c disjoint, AAc=SA\cup A^c=S, so P(A)+P(Ac)=1P(A)+P(A^c)=1.
General addition rule
P(AB)=P(A)+P(B)P(AB)P(A\cup B)=P(A)+P(B)-P(A\cap B).
Why subtract P(AB)P(A\cap B) in the addition rule
The overlap is counted in both P(A)P(A) and P(B)P(B), so once is removed.
Why P(A)1P(A)\le 1 from the axioms
P(Ac)=1P(A)0P(A)1P(A^c)=1-P(A)\ge0\Rightarrow P(A)\le1.
Which theorem justifies empirical probability
Law of Large Numbers — relative frequency converges to true probability.
Gambler's fallacy — the fix
Independent trials have no memory; LLN swamps early data, it does not compensate.

Recall Feynman: explain to a 12-year-old

Probability is a "how-likely score" from 0 (never) to 1 (always). Three ways to get the score: (1) Count — a die has 6 equal sides, so each side scores 1/61/6. (2) Try it lots of times — flip a weird bottle cap 100 times, count landings, that fraction is the score. (3) Rules — a wise mathematician (Kolmogorov) said: scores are never negative, the score of "something happens" is 1, and for things that can't happen together you just add their scores. From these 3 tiny rules you can figure out every probability fact, like magic.

Connections

Concept Map

measured over

subset gives

logic maps to

way 1

way 2

way 3 rigorous

requires

justified by

special case of

special case of

lets us compute

Probability 0 to 1

Sample space S

Event A subset of S

Set algebra: union, intersection, complement

Classical: m over n

Empirical: f over N

Axiomatic: Kolmogorov axioms

Symmetry equally likely

Law of Large Numbers

Hinglish (regional understanding)

Intuition Hinglish mein samjho

Dekho, probability basically ek number hai 0 se 1 ke beech jo batata hai koi cheez kitni "likely" hai. Isse nikalne ke teen tareeke hain. Classical: jab saare outcomes equally likely ho — jaise fair dice ya cards — tab bas favourable ko total se divide kar do, m/nm/n. Lekin ye tabhi valid hai jab symmetry ho; tedha coin par ye fail ho jayega.

Empirical ka matlab hai experiment ko baar-baar karo aur fraction nikalo — thumbtack ya bottle cap jaise cheezon ke liye jahan counting possible nahi. Law of Large Numbers kehta hai ki jaise-jaise trials badhte hain, ye relative frequency f/Nf/N true probability par settle ho jati hai. Isiliye 10 tosses par bharosa mat karo, 1000 par karo.

Phir aaye Kolmogorov — unhone kaha ki chhodo kaise number nikalte ho, bas 3 rules follow hone chahiye: (1) probability negative nahi hoti, (2) P(S)=1P(S)=1 yaani kuch na kuch to hoga hi, (3) jo events ek saath nahi ho sakte unki probability add ho jati hai. In 3 axioms se baaki sab formulae derive ho jaate hain — jaise P(Ac)=1P(A)P(A^c)=1-P(A) aur addition rule P(AB)=P(A)+P(B)P(AB)P(A\cup B)=P(A)+P(B)-P(A\cap B).

Ek common galti: A ya B ke liye seedha add mat karo agar overlap ho — warna ABA\cap B do baar count ho jayega, isliye ek baar minus karna padta hai. Aur gambler's fallacy se bacho: 5 tails aane ke baad head "due" nahi hota, har toss independent hai, memory nahi hoti. Ye foundation aage Bayes' theorem aur distributions tak le jayega, isliye ye pakka karna zaroori hai.

Go deeper — visual, from zero

Test yourself — Statistics & Probability — Intermediate

Connections