Foundations — Sample spaces, events, and axioms of probability

This page assumes you know nothing. We name every symbol the parent note uses, draw the picture it stands for, and say why the topic can't live without it. Read top to bottom — each idea is a brick for the next.

Brick 0: What is a "set"?

Before any probability symbol makes sense, we need one idea: a set.

The picture: imagine a bag. Everything you drop in the bag is an element; the bag itself is the set.

Why the topic needs it: every object in probability — the list of outcomes, the group of outcomes we care about — is a set. If you can't picture a bag of things, none of the later symbols land.

Look at the figure: the big box is one set (all the dots), and a smaller circle inside is another set made only of the dots we chose. That nesting — a set living inside a bigger set — is the whole game.

• $\in$ means is an element of: $H\in \{H,T\}$ reads "$H$ is in the bag."
• $\subseteq$ means is a subset of: every element of the left set is also in the right set. The small circle $\subseteq$ the big box.
• $\varnothing$ is the empty set: the bag with nothing in it, $\{\}$.

Brick 1: The sample space $\Omega$

Now we give the biggest bag a name.

Why the letter $\Omega$? It's just tradition — omega is the last letter of the Greek alphabet, a nod to "everything up to the end." You could call it $S$; the meaning is what matters.

The picture: $\Omega$ is the outer rectangle in every probability diagram — the board on which we draw. A single outcome is one dot on that board.

Why the topic needs it: you cannot ask "what fraction of the time?" until you've drawn "fraction of what?". $\Omega$ is that "what."

• $|\Omega |$ means the size (number of elements) of $\Omega$. Two vertical bars $|\cdot |$ = "how many things are in here." For one die, $|\Omega |=6$.

Brick 2: The outcome $\omega$

The picture: if $\Omega$ is the whole board of dots, $\omega$ is your finger pointing at exactly one dot.

Why the topic needs it: the finest, unsplittable thing we can assign a probability to is a single outcome. Everything bigger is built by grouping outcomes.

Brick 3: An event $E$ (a shaded region)

The picture: draw a loop around some of the dots. The dots inside the loop form the event. The whole board $\Omega$ is itself an event ("something happens"), and the empty loop $\varnothing$ is the impossible event ("nothing happens").

Why subsets and not single dots? Because real questions bundle outcomes. "The die shows an even number" is the loop $\{2,4,6\}$ — three dots at once, not one.

In the figure, the die's six dots sit in the box. The blue loop $A$ = "even" holds $\{2,4,6\}$; the pink loop $B$ = "greater than 3" holds $\{4,5,6\}$. Where the loops overlap sits $\{4,6\}$ — outcomes belonging to both events.

Brick 4: Combining events — the picture-operations

Events are sets, so we combine them with set operations. Each is a picture.

Why we need all three: real questions chain words like "or", "and", "not". Without a symbol for each, we can't translate a sentence into something we can measure.

Two special phrases you'll meet constantly:

• Mutually exclusive (disjoint): $A\cap B=\varnothing$. The loops don't touch — no overlap dots. This is exactly the situation the third axiom below cares about.
• Exhaustive: the loops together cover all of $\Omega$, i.e. their union is $\Omega$.

Brick 5: The probability function $P$

The picture: think of the box $\Omega$ as holding exactly 1 litre of paint spread over its dots. $P(E)$ is how much of that paint sits inside loop $E$. A region covering half the paint has $P=0.5$.

Why the topic needs it: $\Omega$ and events describe what can happen; $P$ is the only thing that says how likely. It turns geometry (regions) into numbers.

• $\forall$ means for all. $\forall E\subseteq \Omega$ reads "for every event $E$ inside $\Omega$."
• $\ge$ means "greater than or equal to"; $\le$ means "less than or equal to."

Brick 6: The three axioms as paint-rules

With the paint picture, the parent note's Kolmogorov axioms become obvious.

The figure shows why the disjoint condition in rule 3 is essential: two separated loops let you add their paint safely, but two overlapping loops would count the overlap twice — which is precisely why the parent note derives inclusion–exclusion to subtract $P(A\cap B)$.

Brick 7: Counting for equally-likely outcomes

When every dot carries the same amount of paint, likelihood becomes pure counting.

Why this is allowed: if all $|\Omega |$ dots share the paint equally, each holds $\frac{1}{|\Omega |}$; a loop with $|E|$ dots holds $|E|\cdot \frac{1}{|\Omega |}$ — that's rule 3 applied to single dots.

The one guardrail: this formula is only valid when the dots are genuinely equally likely (fair die, random draw). It is not a law of nature — "rain or no rain" has two dots but they are not equally painted.

How the bricks feed the topic

Everything upstream of TOPIC is what this D1 page installs; the parent note lives at TOPIC and beyond.

Where these bricks are reused

• Conditional Probability shrinks the box $\Omega$ down to one event and re-measures paint inside it.
• Bayes' Theorem rearranges those paint fractions to flip "given" around.
• Random Variables attach a number to every outcome $\omega$ — a function from $\Omega$ to the reals.
• Probability Distributions describe how the 1 litre of paint is spread.
• Maximum Likelihood Estimation uses the counting formula $\frac{|E|}{|\Omega |}$ in the limit of much data.
• Classification Metrics and Entropy and Information are built entirely from $P$ over events.

Equipment checklist

Answer each aloud before moving on; reveal to check.