Foundations — Joint distributions — joint PMF - PDF, marginal, conditional

Before you can trust the parent note, you must be fluent in every mark on the page. This child builds each symbol from nothing — plain words, then a picture, then why the topic can't live without it. Read top to bottom; each brick sits on the one below.

1. Random variable — $X$, $Y$

Picture it. Roll a die: the machine that will output a number is $X$; the number $3$ it actually shows is $x$. The capital is the slot; the lower-case is a value in the slot.

Why the topic needs it. The whole chapter studies two such slots at once — height $X$ and weight $Y$ — so you must never confuse "the variable" (capital) with "a value of it" (lower-case). Every formula like $P(X=x)$ literally reads "probability the slot $X$ lands on the number $x$."

2. Probability $P(\cdot )$ and the event $\{X=x\}$

Picture it. Imagine 100 repetitions of the experiment as 100 dots. $P(X=x)$ is the fraction of dots for which $X$ came out $x$. $0$ = never, $1$ = always, $0.5$ = half the dots.

Why the topic needs it. A joint PMF is built out of these probabilities — $p_{X,Y}(x,y)=P(X=x\text{ and }Y=y)$. If "probability of an event" is fuzzy, the whole table is fuzzy.

3. "AND" of two events — intersection

Picture it (see figure). Draw the plane with $x$ across and $y$ up. The event $\{X=x\}$ is a whole vertical line; $\{Y=y\}$ is a whole horizontal line. Their AND is the single crossing point where both are satisfied.

Why the topic needs it. This crossing point is one cell of the joint. The joint distribution is just "how much probability sits at every such crossing."

4. Disjoint events and additivity

Picture it. Two blobs that do not overlap. The chance of landing in either blob is simply the two areas summed — no double-counting because nothing is shared.

Why the topic needs it. This is the engine behind marginals. The events $\{X=x,Y=0\}$, $\{X=x,Y=1\}$, $\{X=x,Y=2\},\dots$ are disjoint (only one value of $Y$ can occur), so their probabilities add to give $P(X=x)$. That addition is the marginal formula. This is exactly Law of Total Probability in disguise.

5. Sum notation $\sum$

Picture it. A table row: ${\sum }_y{p}_{X,Y}(x,y)$ = slide along the row, adding each cell. The answer is written in the margin — hence "marginal."

Why the topic needs it. Discrete marginals are row/column sums: $p_X(x)={\sum }_y{p}_{X,Y}(x,y)$.

6. Integral $\int$ and area-under-a-curve

Picture it (see figure). The shaded region under a curve; the $dx$ is one skinny rectangle, and $\int$ sweeps and sums them.

Why the topic needs it. In continuous joint distributions, probability = volume/area under the density, obtained by integrating. Marginals become $f_X(x)=\int f_{X,Y}(x,y)dy$: the continuous version of "add along the row."

7. Density vs. probability — the $f$ symbol

Picture it (see figure). A hill sitting over the plane. Total volume of the whole hill $=1$. Volume over one patch $=$ probability of landing in that patch.

8. The double integral $\iint$ and the support

Picture it. The triangle $0<y<x<1$ from the parent's example is a support: outside it the hill has zero height, so integrals there contribute nothing. Getting the limits of integration right = tracing the edge of this footprint.

Why the topic needs it. Wrong support ⇒ wrong limits ⇒ wrong marginals. The parent's fourth "common mistake" is exactly forgetting this. Always sketch the footprint first.

9. Conditional bar $\mid$ and re-normalization

Picture it. Take the joint hill and cut a single slice at $Y=y$. That slice has some total (its area = $f_Y(y)$), but a valid distribution must total $1$. So you scale the slice up by dividing by $f_Y(y)$. The rescaled slice is the conditional. This rests on Conditional Probability: $P(A\mid B)=P(A\cap B)/P(B)$.

Why the topic needs it. Conditioning is slice + re-normalize, captured by "Conditional $=$ Joint $÷$ Marginal."

10. Marginal $f_X$, and independence factoring

Why the topic needs them. Marginals recover single-variable facts; independence is the special "no interaction" case where the hill is just an outer product of two 1D shapes. Later tools — Covariance and Correlation, Bayes' Theorem, the Bivariate Normal Distribution, and Expectation and Variance over pairs — all build directly on these bricks.

The prerequisite map

Equipment checklist

Self-test: cover the right side and answer each. If any tricks you, re-read its section.