Intuition The big picture (WHY this exists)Before we can compute probabilities, we must agree on what we are even measuring .
A probability space is the rigorous "stage" on which all of probability is performed.
It answers three questions:
What can happen? → the sample space Ω \Omega Ω Which collections of outcomes are we allowed to assign a probability to? → the sigma-algebra F \mathcal{F} F How much probability does each allowed collection get? → the measure P P P
WHY bother with F \mathcal{F} F [ 0 , 1 ] [0,1] [ 0 , 1 ] cannot consistently assign a probability to every subset — pathological sets break the rules. So we restrict to a well-behaved family F \mathcal{F} F
Definition Probability spaceA probability space is a triple ( Ω , F , P ) (\Omega, \mathcal{F}, P) ( Ω , F , P )
Ω \Omega Ω sample space , the set of all possible outcomes ω \omega ω F \mathcal{F} F sigma-algebra (a collection of subsets of Ω \Omega Ω events ).P P P probability measure , a function P : F → [ 0 , 1 ] P:\mathcal{F}\to[0,1] P : F → [ 0 , 1 ]
WHY these three? They are the minimum closure rules so that any logical combination of events ("not A", "A or B", "A and B", "at least one of infinitely many") is still an event we can measure. From 1–3 you get everything else:
∅ = Ω c ∈ F \varnothing = \Omega^c \in \mathcal{F} ∅ = Ω c ∈ F Countable intersections : ⋂ i A i = ( ⋃ i A i c ) c ∈ F \bigcap_i A_i = \big(\bigcup_i A_i^c\big)^c \in \mathcal{F} ⋂ i A i = ( ⋃ i A i c ) c ∈ F
F \mathcal{F} F F \mathcal{F} F list of yes/no questions you are allowed to ask . If you can ask "did A happen?", you must also be able to ask "did A not happen?" and "did A or B happen?". A sigma-algebra is just the rule that this question-list is logically complete.
WHY each axiom?
A1 — probability is a "mass"/"size"; sizes are never negative.A2 — something in Ω \Omega Ω A3 — if you split an event into disjoint pieces, the probabilities add. The "countable" (not just finite) version is what makes limits, integrals, and continuous distributions work.
Nothing below is an extra rule — all are theorems forced by A1–A3.
P ( ∅ ) = 0 P(\varnothing)=0 P ( ∅ ) = 0 A 1 = Ω , A 2 = A 3 = ⋯ = ∅ A_1=\Omega,\ A_2=A_3=\dots=\varnothing A 1 = Ω , A 2 = A 3 = ⋯ = ∅ ⋃ A i = Ω \bigcup A_i=\Omega ⋃ A i = Ω P ( Ω ) = P ( Ω ) + ∑ i ≥ 2 P ( ∅ ) \ P(\Omega)=P(\Omega)+\sum_{i\ge2}P(\varnothing) P ( Ω ) = P ( Ω ) + ∑ i ≥ 2 P ( ∅ ) Why this step? A3 lets me expand the union into a sum of identical terms P ( ∅ ) P(\varnothing) P ( ∅ ) ∑ i ≥ 2 P ( ∅ ) = 0 \sum_{i\ge2}P(\varnothing)=0 ∑ i ≥ 2 P ( ∅ ) = 0 ≥ 0 \ge0 ≥ 0 0 0 0 ■ \blacksquare ■
finite additivity P ( A ∪ B ) = P ( A ) + P ( B ) P(A\cup B)=P(A)+P(B) P ( A ∪ B ) = P ( A ) + P ( B ) A , B A,B A , B A 1 = A , A 2 = B , A 3 = A 4 = ⋯ = ∅ A_1=A,\ A_2=B,\ A_3=A_4=\dots=\varnothing A 1 = A , A 2 = B , A 3 = A 4 = ⋯ = ∅ P ( ∅ ) = 0 P(\varnothing)=0 P ( ∅ ) = 0 P ( A ∪ B ) = P ( A ) + P ( B ) + 0 + 0 + … \ P(A\cup B)=P(A)+P(B)+0+0+\dots P ( A ∪ B ) = P ( A ) + P ( B ) + 0 + 0 + … ■ \blacksquare ■ Why this step? A3 is stated for infinite unions; padding with ∅ \varnothing ∅
Worked example Derive thecomplement rule P ( A c ) = 1 − P ( A ) P(A^c)=1-P(A) P ( A c ) = 1 − P ( A ) A A A A c A^c A c A ∪ A c = Ω A\cup A^c=\Omega A ∪ A c = Ω P ( Ω ) = P ( A ) + P ( A c ) P(\Omega)=P(A)+P(A^c) P ( Ω ) = P ( A ) + P ( A c ) P ( Ω ) = 1 P(\Omega)=1 P ( Ω ) = 1 P ( A c ) = 1 − P ( A ) P(A^c)=1-P(A) P ( A c ) = 1 − P ( A ) ■ \blacksquare ■ Consequence: P ( A ) = 1 − P ( A c ) ≤ 1 P(A)=1-P(A^c)\le 1 P ( A ) = 1 − P ( A c ) ≤ 1 0 ≤ P ( A ) ≤ 1 0\le P(A)\le 1 0 ≤ P ( A ) ≤ 1 [ 0 , 1 ] [0,1] [ 0 , 1 ] derived , not assumed.
monotonicity A ⊆ B ⇒ P ( A ) ≤ P ( B ) A\subseteq B\Rightarrow P(A)\le P(B) A ⊆ B ⇒ P ( A ) ≤ P ( B ) B = A ∪ ( B ∖ A ) B = A \cup (B\setminus A) B = A ∪ ( B ∖ A ) B ∖ A = B ∩ A c B\setminus A = B\cap A^c B ∖ A = B ∩ A c P ( B ) = P ( A ) + P ( B ∖ A ) P(B)=P(A)+P(B\setminus A) P ( B ) = P ( A ) + P ( B ∖ A ) Why this step? Splitting B B B A A A P ( B ∖ A ) ≥ 0 P(B\setminus A)\ge0 P ( B ∖ A ) ≥ 0 P ( B ) ≥ P ( A ) P(B)\ge P(A) P ( B ) ≥ P ( A ) ■ \blacksquare ■
inclusion–exclusion P ( A ∪ B ) = P ( A ) + P ( B ) − P ( A ∩ B ) P(A\cup B)=P(A)+P(B)-P(A\cap B) P ( A ∪ B ) = P ( A ) + P ( B ) − P ( A ∩ B ) A ∪ B = ( A ∖ B ) ⊔ ( A ∩ B ) ⊔ ( B ∖ A ) . A\cup B=(A\setminus B)\ \sqcup\ (A\cap B)\ \sqcup\ (B\setminus A). A ∪ B = ( A ∖ B ) ⊔ ( A ∩ B ) ⊔ ( B ∖ A ) . P ( A ∪ B ) = P ( A ∖ B ) + P ( A ∩ B ) + P ( B ∖ A ) P(A\cup B)=P(A\setminus B)+P(A\cap B)+P(B\setminus A) P ( A ∪ B ) = P ( A ∖ B ) + P ( A ∩ B ) + P ( B ∖ A ) P ( A ) = P ( A ∖ B ) + P ( A ∩ B ) P(A)=P(A\setminus B)+P(A\cap B) P ( A ) = P ( A ∖ B ) + P ( A ∩ B ) P ( B ) = P ( B ∖ A ) + P ( A ∩ B ) P(B)=P(B\setminus A)+P(A\cap B) P ( B ) = P ( B ∖ A ) + P ( A ∩ B ) Why this step? Each of A A A B B B A ∩ B A\cap B A ∩ B P ( A ) + P ( B ) = P ( A ∖ B ) + P ( B ∖ A ) + 2 P ( A ∩ B ) P(A)+P(B)=P(A\setminus B)+P(B\setminus A)+2P(A\cap B) P ( A ) + P ( B ) = P ( A ∖ B ) + P ( B ∖ A ) + 2 P ( A ∩ B ) P ( A ∩ B ) P(A\cap B) P ( A ∩ B ) P ( A ∪ B ) = P ( A ) + P ( B ) − P ( A ∩ B ) . ■ P(A\cup B)=P(A)+P(B)-P(A\cap B).\ \blacksquare P ( A ∪ B ) = P ( A ) + P ( B ) − P ( A ∩ B ) . ■
Worked example Finite (die) spaceRoll a fair die. Ω = { 1 , 2 , 3 , 4 , 5 , 6 } \Omega=\{1,2,3,4,5,6\} Ω = { 1 , 2 , 3 , 4 , 5 , 6 } F = 2 Ω \mathcal{F}=2^{\Omega} F = 2 Ω P ( A ) = ∣ A ∣ / 6 P(A)=|A|/6 P ( A ) = ∣ A ∣/6 A = { even } = { 2 , 4 , 6 } A=\{\text{even}\}=\{2,4,6\} A = { even } = { 2 , 4 , 6 } B = { >3 } = { 4 , 5 , 6 } B=\{\text{>3}\}=\{4,5,6\} B = { >3 } = { 4 , 5 , 6 }
P ( A ) = 3 / 6 P(A)=3/6 P ( A ) = 3/6 P ( B ) = 3 / 6 P(B)=3/6 P ( B ) = 3/6 A ∩ B = { 4 , 6 } A\cap B=\{4,6\} A ∩ B = { 4 , 6 } P ( A ∩ B ) = 2 / 6 P(A\cap B)=2/6 P ( A ∩ B ) = 2/6 P ( A ∪ B ) = 3 6 + 3 6 − 2 6 = 4 6 = 2 3 P(A\cup B)=\tfrac36+\tfrac36-\tfrac26=\tfrac46=\tfrac23 P ( A ∪ B ) = 6 3 + 6 3 − 6 2 = 6 4 = 3 2 Check: A ∪ B = { 2 , 4 , 5 , 6 } A\cup B=\{2,4,5,6\} A ∪ B = { 2 , 4 , 5 , 6 } 4 / 6 4/6 4/6 Why F = 2 Ω \mathcal{F}=2^{\Omega} F = 2 Ω Ω \Omega Ω every subset is measurable — no pathology.
Worked example Continuous (uniform) spacePick a point uniformly in Ω = [ 0 , 1 ] \Omega=[0,1] Ω = [ 0 , 1 ] F = \mathcal{F}= F = P ( [ a , b ] ) = b − a P([a,b])=b-a P ([ a , b ]) = b − a
P ( [ 0 , 0.3 ] ) = 0.3 P([0,0.3])=0.3 P ([ 0 , 0.3 ]) = 0.3 P ( { 0.5 } ) = 0 P(\{0.5\})=0 P ({ 0.5 }) = 0 Why a single point has probability 0: { 0.5 } = ⋂ n [ 0.5 , 0.5 + 1 n ] \{0.5\}=\bigcap_n[0.5,\,0.5+\tfrac1n] { 0.5 } = ⋂ n [ 0.5 , 0.5 + n 1 ] P ( [ 0.5 , 0.5 + 1 n ] ) = 1 n → 0 P([0.5,0.5+\tfrac1n])=\tfrac1n\to0 P ([ 0.5 , 0.5 + n 1 ]) = n 1 → 0 0.5 0.5 0.5 can occur — probability 0 ≠ impossible .
Common mistake "Every subset ofΩ \Omega Ω Why it feels right: On a finite die, it is — F = 2 Ω \mathcal{F}=2^\Omega F = 2 Ω The fix: On uncountable Ω \Omega Ω [ 0 , 1 ] [0,1] [ 0 , 1 ] F \mathcal{F} F smaller than 2 Ω 2^\Omega 2 Ω
Common mistake "Additivity works for any union:P ( A ∪ B ) = P ( A ) + P ( B ) P(A\cup B)=P(A)+P(B) P ( A ∪ B ) = P ( A ) + P ( B ) Why it feels right: It's true when A , B A,B A , B The fix: If they overlap you double-count A ∩ B A\cap B A ∩ B P ( A ∩ B ) P(A\cap B) P ( A ∩ B )
P ( A ) = 0 P(A)=0 P ( A ) = 0 A A A Why it feels right: Impossible events (∅ \varnothing ∅ The fix: The converse fails. In continuous spaces individual points have probability 0 but can still occur. "Probability 0" = "almost never", not "never".
Common mistake "Finite additivity is enough; countable is just fancy."Why it feels right: All real coin/die experiments are finite.
The fix: Countable additivity (A3) is strictly stronger and is what guarantees continuity of P P P
Recall Quick self-test (hide answers)
The three closure rules of a sigma-algebra? → contains Ω \Omega Ω
Why range [ 0 , 1 ] [0,1] [ 0 , 1 ] P P P ≥ 0 \ge0 ≥ 0 ≤ 1 \le1 ≤ 1
State Kolmogorov A3. → countable additivity for disjoint events.
Derive P ( A c ) P(A^c) P ( A c ) 1 − P ( A ) 1-P(A) 1 − P ( A ) P ( A ) + P ( A c ) = P ( Ω ) = 1 P(A)+P(A^c)=P(\Omega)=1 P ( A ) + P ( A c ) = P ( Ω ) = 1
Recall Feynman: explain to a 12-year-oldImagine a big bag (that's Ω \Omega Ω F \mathcal{F} F P P P exactly 1 liter of paint over all the questions: no question gets negative paint (A1), the whole bag uses all 1 liter (A2), and if two questions never overlap, their paint just adds up (A3). Everything else in probability is just pouring this paint around without spilling.
Mnemonic Remember the trio"O-F-P: Outcomes, Filter, Paint."
Ω \Omega Ω O utcomes; F \mathcal{F} F F ilter of askable questions; P P P P aint (total = 1).
And the axioms: "NNA" = N on-negative, N ormalized, A dditive.
What three objects make up a probability space? ( Ω , F , P ) (\Omega,\mathcal{F},P) ( Ω , F , P ) : sample space, sigma-algebra of events, probability measure.
What is the sample space Ω \Omega Ω The set of all possible outcomes of the experiment.
List the three sigma-algebra axioms. Contains
Ω \Omega Ω ; closed under complement; closed under countable unions.
Why must F \mathcal{F} F So any logical combination of events (not, or, and) is still a measurable event.
State Kolmogorov's three axioms. A1
P ( A ) ≥ 0 P(A)\ge0 P ( A ) ≥ 0 ; A2
P ( Ω ) = 1 P(\Omega)=1 P ( Ω ) = 1 ; A3 countable additivity for disjoint events.
Derive P ( ∅ ) = 0 P(\varnothing)=0 P ( ∅ ) = 0 Use disjoint
Ω , ∅ , ∅ , … \Omega,\varnothing,\varnothing,\dots Ω , ∅ , ∅ , … ; A3 gives
P ( Ω ) = P ( Ω ) + ∑ P ( ∅ ) P(\Omega)=P(\Omega)+\sum P(\varnothing) P ( Ω ) = P ( Ω ) + ∑ P ( ∅ ) , so each
P ( ∅ ) = 0 P(\varnothing)=0 P ( ∅ ) = 0 .
Derive the complement rule. A ∪ A c = Ω A\cup A^c=\Omega A ∪ A c = Ω disjoint ⇒
P ( A ) + P ( A c ) = 1 P(A)+P(A^c)=1 P ( A ) + P ( A c ) = 1 ⇒
P ( A c ) = 1 − P ( A ) P(A^c)=1-P(A) P ( A c ) = 1 − P ( A ) .
Why is 0 ≤ P ( A ) ≤ 1 0\le P(A)\le1 0 ≤ P ( A ) ≤ 1 ≥ 0 \ge0 ≥ 0 is A1;
≤ 1 \le1 ≤ 1 follows from complement rule + A2. The range is derived.
State and derive inclusion–exclusion for two events. P ( A ∪ B ) = P ( A ) + P ( B ) − P ( A ∩ B ) P(A\cup B)=P(A)+P(B)-P(A\cap B) P ( A ∪ B ) = P ( A ) + P ( B ) − P ( A ∩ B ) , by splitting into disjoint pieces
A ∖ B , A ∩ B , B ∖ A A\setminus B, A\cap B, B\setminus A A ∖ B , A ∩ B , B ∖ A .
Why can't F = 2 Ω \mathcal{F}=2^\Omega F = 2 Ω [ 0 , 1 ] [0,1] [ 0 , 1 ] Non-measurable (Vitali) sets exist that violate countable additivity, so we use Borel sets.
Does P ( A ) = 0 P(A)=0 P ( A ) = 0 A A A No. In continuous spaces single points have probability 0 yet can occur ("almost never").
Monotonicity: if A ⊆ B A\subseteq B A ⊆ B P ( A ) ≤ P ( B ) P(A)\le P(B) P ( A ) ≤ P ( B ) , since
P ( B ) = P ( A ) + P ( B ∖ A ) P(B)=P(A)+P(B\setminus A) P ( B ) = P ( A ) + P ( B ∖ A ) and
P ( B ∖ A ) ≥ 0 P(B\setminus A)\ge0 P ( B ∖ A ) ≥ 0 .
Why countable (not just finite) additivity? It gives continuity of
P P P and lets us handle limits, integrals, and continuous distributions.
Closed under countable union
Countable intersection derived
Intuition Hinglish mein samjhoDekho, probability ko rigorous banane ke liye sabse pehle ek "stage" chahiye hota hai jise hum probability space ( Ω , F , P ) (\Omega, \mathcal{F}, P) ( Ω , F , P ) Ω \Omega Ω sample space — saare possible outcomes ka set (jaise dice me { 1 , 2 , 3 , 4 , 5 , 6 } \{1,2,3,4,5,6\} { 1 , 2 , 3 , 4 , 5 , 6 } F \mathcal{F} F sigma-algebra hai — yeh un saare sawaalon ki list hai jinke baare me hum "haan ya na" pooch sakte hain (events). Aur P P P measure hai jo har event ko ek number [ 0 , 1 ] [0,1] [ 0 , 1 ]
Sigma-algebra ke teen rule hain: Ω \Omega Ω [ 0 , 1 ] [0,1] [ 0 , 1 ]
Kolmogorov ke teen axioms yaad rakho — NNA : Non-negative (P ( A ) ≥ 0 P(A)\ge0 P ( A ) ≥ 0 P ( Ω ) = 1 P(\Omega)=1 P ( Ω ) = 1 derive hoti hai — P ( ∅ ) = 0 P(\varnothing)=0 P ( ∅ ) = 0 P ( A c ) = 1 − P ( A ) P(A^c)=1-P(A) P ( A c ) = 1 − P ( A ) P ( A ∪ B ) = P ( A ) + P ( B ) − P ( A ∩ B ) P(A\cup B)=P(A)+P(B)-P(A\cap B) P ( A ∪ B ) = P ( A ) + P ( B ) − P ( A ∩ B ) [ 0 , 1 ] [0,1] [ 0 , 1 ]
Do common galtiyan avoid karo: (1) overlap wale events me direct add mat karo — A ∩ B A\cap B A ∩ B P ( A ) = 0 P(A)=0 P ( A ) = 0
