Transformations of random variables — change-of-variable technique: Visual walkthrough

Step 0 — The words we must agree on first

Before any formula, four plain-language ideas. We will never use them without this meaning.

Density-per-length is the crucial one. Density is not a probability. You only get a probability when you multiply density by a length — that's the jam over a slab.

Step 1 — Probability is jam, and jam is conserved

WHAT. We spread a fixed total amount of jam (total probability $= 1$ ) along the $x$ -ruler. The height of the jam at point $x$ is the density $f_{X} (x)$ .

WHY. Everything downstream is one idea: smearing the ruler cannot create or destroy jam. If we later stretch the ruler, the same jam just sits on a longer or shorter stretch — thinner or thicker — but the amount is untouched.

PICTURE. Figure s01 shows the lavender bell-shaped density $f_{X}$ over the $x$ -ruler, with one narrow coral slab standing on $[x, x + d x]$ ; arrows label its height $f_{X} (x)$ and its width $d x$ , and a caption reminds us the total shaded area is 1. Over that tiny slab, the jam is a thin rectangle: $height = thickness of jam f_{X} (x) \times width of the slab d x = mass in that slab .$ Each symbol: $f_{X} (x)$ is how tall the jam stands at $x$ ; $d x$ is how wide the little slab is; their product is the area of the sliver = the probability of landing in $[x, x + d x]$ .

Figure — Transformations of random variables — change-of-variable technique

Step 2 — The machine $g$ moves each slab to a new place

WHAT. The machine sends the point $x$ to the point $y = g (x)$ . So the slab $[x, x + d x]$ gets carried over and lands as a slab $[y, y + d y]$ on the $y$ -ruler.

WHY. We want the jam thickness on the $y$ -ruler, so we must literally follow where each slab of $x$ -jam goes.

PICTURE. Figure s02 draws the curve $y = g (x)$ in lavender; a coral slab of width $d x$ sits on the $x$ -axis, dashed lines rise to the curve and cross to the $y$ -axis, casting a mint slab of width $d y$ ; a note points out that where the curve is steep, $d y > d x$ . Trace up from the little $x$ -slab, hit the curve, trace across: that horizontal shadow is the slab $[y, y + d y]$ . Its width $d y$ is generally different from $d x$ — that difference is the entire story. $d y = slope of the machine g^{'} (x) d x .$ Here $g^{'} (x)$ is the steepness of the curve: a steep curve ( $∣ g^{'} ∣$ big) turns a small $d x$ into a big $d y$ — the slab gets stretched wide.

Step 3 — Same jam, new width ⇒ new thickness

WHAT. The jam that was in $[x, x + d x]$ is exactly the jam now in $[y, y + d y]$ . Set the two masses equal: $mass after f_{Y} (y) ∣ d y ∣ = mass before f_{X} (x) ∣ d x ∣ .$ Each symbol: $f_{Y} (y)$ is the unknown thickness on the new ruler (the density of $Y$ we defined in Step 0); $∣ d y ∣$ its slab width; on the right, the old thickness times the old width. Equal, because it's the same jam.

WHY the absolute values? Width is a length — always positive. If the machine happens to run "backwards" (a decreasing $g$ ), the raw $d y$ could come out negative just from bookkeeping. Wrapping both in $∣ \cdot ∣$ keeps us honest: we compare sizes of slabs, not signed ones.

PICTURE. Figure s03 places two rectangles of equal area side by side: a tall narrow coral one (the $x$ -slab) and a short wide mint one (the $y$ -slab), with a lavender arrow marked $g$ carrying one to the other and the equation $f_{Y} (y) ∣ d y ∣ = f_{X} (x) ∣ d x ∣$ on top. Equal area = conserved jam.

Solve for the unknown thickness by dividing: $f_{Y} (y) = f_{X} (x) \frac{d x}{d y} .$ The ratio $\frac{d x}{d y}$ is the squeeze factor: old width divided by new width. Wide new slab ( $∣ d y ∣$ big) ⇒ small factor ⇒ thin jam.

Step 4 — But $y$ names the point, not $x$ — invert the machine

WHAT. The formula still mentions $x$ . On the $y$ -ruler we only know $y$ . So replace $x$ by whatever $x$ produced this $y$ : $x = g^{- 1} (y)$ .

WHY this tool — the inverse? We need "which input gave this output?". That question is exactly what an inverse function answers: $g^{- 1}$ undoes $g$ . No other tool answers "run the machine backwards". This is the same logical move as Inverse Transform Sampling and the Cumulative Distribution Function-based derivation in the parent.

PICTURE. Figure s04 shows the same lavender curve, but read backwards: a dashed mint line starts at a chosen $y$ on the vertical axis, meets the curve, and a coral line drops to the $x$ -axis, marking the landing point $x = g^{- 1} (y)$ .

WHY the squeeze factor equals $1/∣ g^{'} (x) ∣$ . We need $\frac{d}{d y} g^{- 1} (y)$ , but we only easily know the slope of $g$ itself, $g^{'} (x)$ . They are reciprocals, and here is the one-line reason. By definition the inverse undoes the machine, so for every $y$ : $g (g^{- 1} (y)) = y .$ Differentiate both sides with respect to $y$ . The right side gives $1$ . The left side, by the chain rule (differentiate the outer $g$ at the inner point, times the derivative of the inner), gives $g^{'} (g^{- 1} (y)) \cdot \frac{d}{d y} g^{- 1} (y)$ . Setting them equal:

\quad\Longrightarrow\quad \frac{d}{dy}g^{-1}(y)=\frac{1}{g'(x)},\ \ x=g^{-1}(y).$$ So the two slopes multiply to $1$ — each is the reciprocal of the other. That is *why* a steep machine ($|g'|$ big) gives a small squeeze factor: stretching the slab wide dilutes the jam. This step needs $g'(x)\ne0$ — exactly the regularity we assumed in Step 0. Substituting $x=g^{-1}(y)$ into the Step 3 result and writing the squeeze factor as the derivative of the inverse: > [!formula] The result, assembled (one-to-one $g$) > $$\boxed{\,f_Y(y)=f_X\!\big(g^{-1}(y)\big)\,\left|\dfrac{d}{dy}\,g^{-1}(y)\right|\,} > \;=\;f_X\!\big(g^{-1}(y)\big)\,\dfrac{1}{\big|g'(g^{-1}(y))\big|}$$ > - $g^{-1}(y)$ — "which $x$ made this $y$", the point on the old ruler. > - $f_X(\cdots)$ — the old jam thickness read at that point. > - $\left|\frac{d}{dy}g^{-1}(y)\right|=\left|\frac{dx}{dy}\right|=\frac{1}{|g'(x)|}$ — the squeeze factor, how much the slab shrank. > > This form is valid **when $g$ is one-to-one** on the support. The many-to-one form is Step 7's sum. --- ## Step 5 — The two directions of the machine (both cases, drawn) **WHAT.** A monotonic machine runs one of two ways: always up (increasing $g$) or always down (decreasing $g$). We must check the formula survives both. **WHY.** If we only checked increasing $g$ we'd have a hidden bug for every decreasing map (like $Y=-X$ or $Y=e^{-X}$). The contract: cover *every* case. **PICTURE.** *Figure s05 is a side-by-side pair. Left: a mint straight line sloping up — increasing $g$, "order kept", $dx/dy>0$. Right: a coral straight line sloping down — decreasing $g$, "ruler reversed", $dx/dy<0$, needing the absolute value.* - **Increasing (left).** As $x$ grows, $y$ grows. Slab order is preserved; $\frac{dx}{dy}>0$; the $|\cdot|$ does nothing. - **Decreasing (right).** As $x$ grows, $y$ *shrinks*. The slab is flipped end-for-end; $\frac{dx}{dy}<0$. Raw, this would give a **negative** thickness — nonsense. The $|\cdot|$ flips the sign back, and thickness stays positive. ![[deepdives/dd-maths-4.9.14-d2-s05.png]] > [!mistake] Thinking the sign matters > The sign of $\frac{dx}{dy}$ only records "did the ruler flip?". Jam thickness can never be negative, so we take the size. That is *why* the boxed formula wears absolute-value bars — it silently handles both pictures above at once. --- ## Step 6 — Degenerate case: a flat spot ($g'=0$) **WHAT.** Suppose at some point the machine is momentarily flat: $g'(x)=0$, so $\frac{dx}{dy}=\frac{1}{g'(x)}\to\infty$. **WHY show it.** A reader will eventually meet a $g$ with a flat spot and panic at the "infinity". They shouldn't. **PICTURE.** *Figure s06 shows the lavender parabola $y=g(x)$ that is flat at $x=0$; a wide coral band of $x$-values near $0$ is shaded, with arrows noting they all crush onto $y\approx0$ so the density spikes there.* A whole band of $x$-values gets squashed onto essentially one $y$-value. All their jam **piles up** at that $y$ — the density there genuinely spikes toward infinity. That's not a bug; it's a real pile-up (a spike you can still integrate to a finite mass). ![[deepdives/dd-maths-4.9.14-d2-s06.png]] We saw a live example in the parent: $Y=X^2$ with $X\sim\text{Uniform}(0,1)$ gives $f_Y(y)=\frac{1}{2\sqrt y}$, which blows up at $y\to 0$ exactly because $g'(0)=0$ squashes many small $x$'s onto $y\approx 0$. --- ## Step 7 — Folding case: the machine sends two $x$'s to one $y$ **WHAT.** If $g$ is not one-to-one (e.g. $g(x)=x^2$), two different slabs land on the *same* $y$-slab. Their jam layers **stack**. The one-to-one boxed result of Step 4 no longer applies as-is; it becomes a **sum, one term per source**: $$\boxed{\,f_Y(y)=\sum_i f_X(x_i)\,\left|\frac{dx_i}{dy}\right|\,},\qquad x_i:\ g(x_i)=y.$$ Symbol by symbol: the sum runs over every input $x_i$ that produces this $y$; each brings its own old thickness $f_X(x_i)$ times its own squeeze factor $\left|dx_i/dy\right|=1/|g'(x_i)|$. When there is only one root, the sum has one term and collapses back to Step 4's box — so this is the *general* form and Step 4 is its special case. **WHY add?** Mass conservation is per-*source*: two independent strips of $x$-jam pour into the same $y$-bin, so total = pour 1 + pour 2. **PICTURE.** *Figure s07 shows the lavender parabola $y=x^2$ folding the negative and positive $x$-axes onto the same $y$; dashed lines from a chosen $y$ meet both roots $-\sqrt y$ (coral) and $+\sqrt y$ (mint), with a caption that the two layers add.* This is the machinery behind the [[Chi-square Distribution]] ($X\sim$ [[Normal Distribution]], $Y=X^2$). ![[deepdives/dd-maths-4.9.14-d2-s07.png]] For $X\sim N(0,1)$, both roots $\pm\sqrt y$ give equal thickness (the bell is symmetric), so we get $2\times\frac{1}{\sqrt{2\pi}}e^{-y/2}\cdot\frac{1}{2\sqrt y}=\frac{1}{\sqrt{2\pi y}}e^{-y/2}$ — exactly $\chi^2_1$. --- ## The one-picture summary *Figure s08 compresses the whole derivation onto one canvas: a tall narrow coral $x$-slab, a lavender arrow marked $g$ (stretch) carrying it, a short wide mint $y$-slab of the same area, the boxed formula $f_Y(y)=f_X(g^{-1}(y))\,|dx/dy|$ across the top, and a footer listing the three special cases: fold ⇒ add layers, flat spot ⇒ spike, $|\cdot|$ ⇒ keeps it positive.* ![[deepdives/dd-maths-4.9.14-d2-s08.png]] > [!recall]- Feynman retelling — the whole walkthrough in plain words > Smear a fixed blob of jam along a rubber ruler; how tall it stands at each mark is the density $f_X$. A machine $g$ stretches the ruler into a new shape, and the jam's height on the new ruler is $f_Y$. Wherever the machine is steep it pulls a little piece of ruler into a *wide* stretch, so the same jam now spreads thin; wherever it's gentle the jam bunches up thick. The jam never changes amount — only thickness — so the new thickness equals the old thickness times how much that spot got squeezed, $|dx/dy|$; and that squeeze factor is $1/|g'|$, the reciprocal of the machine's slope (because running forward then backward must return you to where you started, their slopes multiply to one). Because a new mark $y$ doesn't remember which old mark $x$ made it, we run the machine backwards with $g^{-1}$ to find the source. If the machine ran downhill it flips the ruler, which only threatens to make thickness negative, so we take the size with absolute-value bars. If the machine is flat somewhere it crushes a whole band onto one mark and the jam spikes there. And if it folds two marks onto one, we just add the two jam layers. That is the entire change-of-variable technique — one law: jam is conserved. > [!recall] Quick self-check > Why is the factor $\left|\tfrac{dx}{dy}\right|$ and not $\left|\tfrac{dy}{dx}\right|$? ::: We need old-width per new-width to get *new* thickness; that ratio is $dx/dy$, the reciprocal of the slope $g'(x)$. > Why does $\tfrac{d}{dy}g^{-1}(y)=1/g'(x)$? ::: Differentiate $g(g^{-1}(y))=y$ by the chain rule; the two slopes multiply to 1, so each is the reciprocal of the other. > In a fold, why add instead of average? ::: Two independent strips pour their whole mass into the same bin; total mass is the sum, not the mean. > What makes the density spike at a flat spot? ::: $g'\!=\!0$ squashes many $x$'s onto one $y$, piling all their jam there so thickness $\to\infty$. --- ## #flashcards/maths Which slab-width goes on top of the ratio in the squeeze factor? ::: The old width $dx$; the factor is $|dx/dy|$ = old-width per new-width. What geometric quantity is equal on the $x$-side and the $y$-side? ::: The area of the jam rectangle (thickness × width) — the conserved probability mass. When $g$ is decreasing, what does the absolute value fix? ::: A sign flip in $dx/dy$ that would otherwise give a negative (impossible) density. Why substitute $x=g^{-1}(y)$ at the end? ::: On the $y$-ruler we only know $y$; the inverse tells us which $x$ produced it so we can read the old thickness. What happens to the density where $g'(x)=0$? ::: It spikes toward infinity — a band of $x$-values is crushed onto one $y$, piling up jam. What regularity must $g$ have for the 1D formula to hold? ::: Continuous, differentiable, piecewise strictly monotonic, with $g'\ne0$ (nonzero slope) on each invertible stretch.