Probability mass and density functions: Visual walkthrough

Prerequisites we lean on: 1.3.01-Random-variables-and-distributions (what a random variable is), and we finish by touching 1.3.07-Cumulative-distribution-functions.

Step 1 — A random variable is a labelling machine

WHAT. A random variable is just a rule that reads an outcome of some experiment and prints a number. We write it $X$ . If the experiment is "roll a die," then $X$ might print the face value.

WHY. We can't do arithmetic on the word "heads" or on a die-face drawing. Turning every outcome into a number lets us add, average, and plot. Everything below is arithmetic on the printed numbers.

PICTURE. On the left are raw outcomes (the little icons). The machine $X$ maps each to a spot on the number line on the right.

Figure — Probability mass and density functions

Step 2 — The PMF: stack the chances of every outcome that prints the same number

WHAT. For a discrete variable (finitely many, or countably many, separated printed values) the probability mass function is $p_{X} (x) = P (X = x) = all outcomes printing x ω : X (ω) = x \sum each one’s chance P (ω) .$

Reading the equation left to right: $p_{X} (x)$ is the number we want — the chance the machine prints exactly $x$ . The sum $\sum_{ω : X (ω) = x}$ says "walk through every raw outcome whose print is $x$ ." $P (ω)$ is the chance of that one raw outcome. We add them.

WHY add? Because those raw outcomes are different, non-overlapping events. The additivity axiom of probability says the chance of "this OR that OR ..." (for disjoint events) is the sum of chances. Nothing fancier is allowed.

PICTURE. Two dice, $X =$ sum. Six different rolls print $7$ . Their six chances (each $\frac{1}{36}$ ) stack into one bar of height $\frac{6}{36} = \frac{1}{6}$ .

Step 3 — What breaks when values become continuous

WHAT. Now let $X$ be a continuous measurement — a waiting time, a height, a house price. Ask: "What is $P (X = 170.000 \dots)$ , exactly?"

WHY it breaks. There are infinitely many possible values crammed into any interval. If each single value had a positive chance $p > 0$ , then adding up infinitely many of them would blow past $1$ — impossible. The only escape: each single point must have chance $0$ . $P (X = c) = 0 for every single value c .$

PICTURE. As we allow finer and finer values, the PMF bars shrink toward the floor. A bar chart of zeros tells us nothing. We need a new object.

Step 4 — Rescue plan: measure chance per unit length (density)

WHAT. Instead of asking about single points, chop the number line into thin bins of width $Δ x$ . Each bin does have positive chance. Define a new quantity — height so that height × width = bin chance: $f_{X} (x) \approx \frac{P ( x \leq X \leq x + Δ x )}{Δ x} = \frac{chance in the bin}{width of the bin} .$

Term by term: the numerator is the honest, positive chance of landing in the bin. Dividing by the width $Δ x$ gives chance per unit length — a density. That is what $f_{X}$ is.

WHY divide? Because the bare bin-chance shrinks to $0$ as bins get thin, but the ratio chance/width settles onto a stable, finite number. Dividing out the width cancels the shrinking, exposing the shape that survives.

PICTURE. A histogram where the area of each bar equals its chance. Halving the bin width halves each bar's chance but keeps the height (density) roughly fixed — the outline stabilises.

Step 5 — Let the bins vanish: the sum becomes an integral

WHAT. The chance of a whole interval $[a, b]$ is the sum of its bins' chances. Each bin's chance is $f_{X} (x_{i}) Δ x$ (height × width, from Step 4). So $P (a \leq X \leq b) = \sum_{x_{i} \in [a, b]} height f_{X} (x_{i}) width Δ x Δ x \to 0 \int_{a}^{b} f_{X} (x) d x .$

WHY an integral, and not just a sum? As $Δ x \to 0$ we have infinitely many infinitely thin bars. "Sum of height × width, as width → 0" is exactly the definition of the Riemann integral. We didn't invent calculus here — we bumped into it. The $\int$ symbol is a stretched "S" for "Sum," and $d x$ is the vanished $Δ x$ .

PICTURE. Bars fuse into a smooth region; the staircase of rectangles becomes the shaded area under the curve between $a$ and $b$ .

Step 6 — The CDF ties both worlds together

WHAT. Define the Cumulative Distribution Function — "chance of landing at or below $x$ ": $F_{X} (x) = P (X \leq x) = \int_{- \infty}^{x} f_{X} (t) d t .$ Here $t$ is a dummy sweeping variable that runs from $- \infty$ up to the ceiling $x$ ; it just names the position we integrate over so the ceiling $x$ stays free.

WHY. $F_{X}$ is the running total of density from the far left up to $x$ — a probability you can read off directly (unlike density). Set the ceiling $a$ and $b$ and subtract to recover any interval: $P (a \leq X \leq b) = F_{X} (b) - F_{X} (a)$ .

Undo it. By the Fundamental Theorem of Calculus, the density is the slope of the running total: $f_{X} (x) = \frac{d}{d x} F_{X} (x) .$ Where $F_{X}$ rises steeply (probability piling up fast), $f_{X}$ is tall. Where $F_{X}$ is flat, $f_{X} \approx 0$ .

PICTURE. Top: the density $f_{X}$ (a bump). Bottom: its running-area $F_{X}$ , an S-shape climbing from $0$ to $1$ ; the steepest part of the S sits under the peak of the bump.

More on this object lives in 1.3.07-Cumulative-distribution-functions.

Step 7 — Two edge cases you must be able to draw

WHAT. Test the machinery on the simplest continuous law and a decaying one.

(a) Uniform on $[0, 1]$ . Equal chance everywhere means constant density $f_{X} (x) = c$ on $[0, 1]$ . Area must be $1$ : $c \cdot (1 - 0) = 1 \Rightarrow c = 1$ . So $f_{X} = 1$ on $[0, 1]$ , $0$ outside. Then $P (0.3 \leq X \leq 0.7) = 1 \cdot (0.7 - 0.3) = 0.4$ .

(b) Exponential (waiting time), rate $λ > 0$ . CDF $F_{X} (x) = 1 - e^{- λ x}$ for $x \geq 0$ . Differentiate: $f_{X} (x) = λ e^{- λ x}$ . It's tallest at $x = 0$ (short waits common) and decays (long waits rare). Total area: $\int_{0}^{\infty} λ e^{- λ x} d x = 1$ .

WHY these two. The uniform shows a density that isn't a probability yet integrates to $1$ (here it happens to equal $1$ , a coincidence of unit width). The exponential shows a density that starts above the range of a PMF is irrelevant — density $λ$ can be $> 1$ if $λ > 1$ , yet the total area is still exactly $1$ .

PICTURE. Left: the flat uniform box, shaded region = $0.4$ . Right: the exponential's decay, whole area = $1$ .

The one-picture summary

The whole journey on one canvas: discrete bars (chance you can point at) → shrink the bins → a smooth density curve whose shaded area is the chance of an interval → its running total climbs the S-shaped CDF from $0$ to $1$ .

Recall Feynman retelling — say it back in plain words

A random variable is a machine that turns each outcome into a number. For a die you can point at a bar: "sum-of-seven happens $1/6$ of the time," because six different rolls print seven and you add their chances. But when the number can be anything — a height, a wait — no single value can keep a positive chance, or the infinitely many of them would burst past $1$ . So each exact point has chance zero. To recover something useful, I chop the line into thin bins, measure each bin's honest chance, and divide by the bin's width. That ratio — chance per unit length — is the density. As the bins shrink to nothing, "add up height times width" becomes an integral, and the density becomes a smooth curve. The area under that curve over an interval is the interval's probability; the whole area is $1$ . Sweeping the area from the far left up to a moving ceiling gives the CDF, and the density is just how steeply that running total climbs. Test it: a flat box on $[0, 1]$ (uniform) and a decaying $λ e^{- λ x}$ (waiting times) both wrap up exactly one unit of area — and both densities are allowed to poke above $1$ , because a density was never a probability in the first place.

Recall Quick self-check

Why can a PDF value exceed 1 but a PMF value cannot? ::: A PMF value is a probability (capped at 1); a PDF value is a rate (chance per unit length) and only becomes a probability after multiplying by a width. How does the sum in the PMF become the integral in the PDF? ::: Bin-chance = density × width; summing height×width as width→0 is the definition of the Riemann integral. What is $P (X = c)$ for continuous $X$ , and why? ::: 0, because $\int_{c}^{c} f_{X} d x$ has zero width and hence zero area. How do you get the PDF back from the CDF? ::: Differentiate: $f_{X} (x) = \frac{d}{d x} F_{X} (x)$ — the density is the slope of the running total.

Related uses: this density-vs-mass split powers 1.3.08-Expected-value-and-variance, the catalogue in 1.3.09-Common-probability-distributions, 1.4.02-Maximum-likelihood-estimation, the class probabilities of 2.1.05-Softmax-activation, and 3.2.04-Gaussian-processes.