Foundations — Transformations of random variables — change-of-variable technique

This page assumes you have seen none of the notation in the parent note. We build each symbol, anchor it to a picture, and only then let the next symbol lean on it.

1. Random variable $X$ — a number that comes out of a random process

Picture. Think of a machine that spits out a number every time you press a button — sometimes $0.3$, sometimes $0.9$, sometimes $0.51$. The machine is the random process; the number it prints is $X$.

Why the topic needs it. The whole subject is "I know how one machine's numbers behave; I feed those numbers into a function; how does the new machine behave?" You cannot even ask that question without a name for the number: $X$.

• Capital $X$ = the machine / the number before you know which value came out.
• Lowercase $x$ = a specific value the machine could print (a fixed point on the number line).

Keeping these two apart is the single most useful habit. $P(X\le x)$ reads "the chance the machine prints something at or below the fixed spot $x$."

2. The number line and the support — where the paint can land

Picture (figure above). A horizontal ruler. The shaded band is where values are possible; outside it, the machine never prints, so there is zero paint.

Why the topic needs it. After you bend the axis, the new variable can only land where the old one could. If $X$ lives on $(0,1)$ and $Y=X^2$, then $Y$ can only be in $(0,1)$ too. Forgetting the support is one of the four classic mistakes in the parent note.

3. Probability density $f_X$ — how thick the paint is

Picture (figure above). A curve sitting above the ruler. Its height at a point $x$ is $f_X(x)$. Where the curve is tall, paint is thick (values there are more likely); where it is low, paint is thin.

Why "per unit length" matters. Height alone is meaningless — you must multiply by a width to get an amount of paint. That is the next idea and it is the heart of the whole technique.

4. The differential $dx$ — an infinitely thin slab of width

Picture. Zoom into the ruler until a slab $[x,x+dx]$ is so narrow the density curve over it looks flat. Its area is a thin rectangle: $\text{(amount of paint on the slab)}=\underset{\text{height}}{\underbrace{f_X(x)}}\times \underset{\text{width}}{\underbrace{dx}}.$

Why the topic needs it. The entire change-of-variable equation is $f_Y(y)|dy|=f_X(x)|dx|,$ i.e. "amount of paint on the after-slab = amount on the before-slab." Every symbol in that line now has a picture.

5. The function $Y=g(X)$ — bending the ruler

Picture (figure above). Two parallel rulers: the bottom is the $x$-axis, the top is the $y$-axis. The curve $g$ maps each bottom point up to a top point. A slab of width $dx$ on the bottom gets carried to a slab of width $dy$ on the top — and it is usually a different width. That difference in width is exactly why the paint thickness changes.

• Monotonic = the curve only goes up (increasing) or only goes down (decreasing); it never turns around. Then each $y$ comes from exactly one $x$.
• Non-monotonic = the curve turns (like $g(x)=x^2$, a valley), so one $y$ can come from two different $x$ values.

6. The inverse $g^{-1}$ — reading the map backwards

Picture. On the two-ruler figure, pick a point $y$ on top and slide back down the arrow to find the $x$ it came from. That backward slide is $g^{-1}$.

Why the topic needs it. The final formula is written in terms of $y$ (we want to know about $Y$), but $f_X$ only knows about $x$. To evaluate $f_X$ at "the $x$ that produced this $y$", we must write that $x$ as $g^{-1}(y)$. A monotonic $g$ guarantees exactly one such $x$, which is why monotonic is the clean case.

7. The derivative $\frac{dx}{dy}$ — the stretch factor (the Jacobian)

Picture (figure above). A before-slab of width $dx$ on the bottom ruler and the after-slab of width $dy$ it maps to on the top. If the after-slab is twice as wide, the paint on it is half as thick. The ratio $\left|\frac{dx}{dy}\right|$ tells you exactly by how much the thickness rescales.

Why the absolute value. Widths are lengths — always positive. If $g$ decreases, sliding forward in $y$ means sliding backward in $x$, so $\frac{dx}{dy}$ comes out negative. Wrapping it in $|\cdot |$ throws away that direction and keeps only the size of the stretch, so density stays positive.

Why the reciprocal warning. $\frac{dx}{dy}$ and $\frac{dy}{dx}=g^{\prime }(x)$ are upside-down versions of each other: $\left|\frac{dx}{dy}\right|=\frac{1}{|g^{\prime }(x)|}.$ The formula needs the inverse map's slope, $\frac{dx}{dy}$. Grabbing $g^{\prime }(x)$ by reflex is the second classic mistake.

8. The CDF $F_X$ — paint counted from the left

Picture. Sweep a broom from far left up to the point $x$; $F_X(x)$ is how much paint you have swept up. It starts at $0$ (nothing swept) and rises to $1$ (all swept).

Why the topic needs it. Densities can be slippery to manipulate directly, but the CDF always behaves. The parent's cleanest derivation goes $F_Y(y)=P(Y\le y)=P(g(X)\le y)$, translates the event into an $x$-statement, then differentiates to get back the density — because density is the slope of the CDF: $f_X(x)=\frac{d}{dx}{F}_X(x).$

9. Total probability equals one — the sanity check

Why the topic needs it. After you compute a new $f_Y$, integrating it must again give $1$. If you forgot the Jacobian, it usually won't — so this is your built-in error detector.

How the foundations feed the topic

Read it as: the density gives a slab of mass; the transformation and its inverse tell you where the slab goes; the Jacobian tells you how much it rescales; conservation of that mass is the technique.

Equipment checklist

Test yourself — each line reveals its answer.