Transformations of random variables — change-of-variable technique

---

WHY do we need this?

We constantly build new randomness from old: square it ($Y=X^2$ for energy), scale it ($Y=\sigma X+\mu$ to standardize), invert it, log it. Knowing $f_X$ is useless for answering questions about $Y$ unless we can transport the density across $g$.

WHAT is being conserved?

Probability mass, not density. A tiny slab of probability sitting on $[x, x+dx]$ has mass $f_X(x)\,dx$. After the map it lives on $[y, y+dy]$. The masses must match: $\underbrace{f_Y(y)\,|dy|}_{\text{mass after}} = \underbrace{f_X(x)\,|dx|}_{\text{mass before}}$

That single equation is the whole technique.

---

Derivation from scratch (monotonic $g$)

Let $Y=g(X)$ with $g$ strictly monotonic (so it has an inverse $x=g^{-1}(y)$). The cleanest derivation goes through the CDF, because CDFs always work.

Case: $g$ increasing. Then $g(X)\le y \iff X \le g^{-1}(y)$, so $F_Y(y)=P(Y\le y)=P(X\le g^{-1}(y))=F_X\big(g^{-1}(y)\big).$ Differentiate using the chain rule (Why this step? density is the derivative of the CDF): $f_Y(y)=f_X\big(g^{-1}(y)\big)\cdot \frac{d}{dy}g^{-1}(y).$

Case: $g$ decreasing. Now $g(X)\le y \iff X\ge g^{-1}(y)$ (the inequality flips), so $F_Y(y)=1-F_X\big(g^{-1}(y)\big), \quad f_Y(y)=-f_X\big(g^{-1}(y)\big)\frac{d}{dy}g^{-1}(y).$ Here $\frac{d}{dy}g^{-1}<0$, so the minus sign makes the density positive.

Both cases collapse into one using absolute value:

---

Non-monotonic $g$: sum over branches

If $g$ is not one-to-one, several $x$-values map to the same $y$. Each preimage $x_i = g_i^{-1}(y)$ contributes its own slab of mass, so we add them: $f_Y(y)=\sum_{i}\, f_X\big(x_i\big)\,\left|\frac{dx_i}{dy}\right|, \qquad x_i \text{ ranges over all roots of } g(x)=y.$

---

Worked examples

---

Bivariate / general case (Jacobian matrix)

For $(U,V)=g(X,Y)$ invertible with inverse $(x,y)=h(u,v)$:

J=\begin{pmatrix}\partial x/\partial u & \partial x/\partial v\\ \partial y/\partial u & \partial y/\partial v\end{pmatrix}.$$ Same conservation idea: $|\det J|$ is the **area-scaling factor** of the map (a 2D stretch). --- ## Common mistakes > [!mistake] Forgetting the Jacobian entirely > **Wrong idea:** "Just plug $x=g^{-1}(y)$ into $f_X$: $f_Y(y)=f_X(g^{-1}(y))$." > **Why it feels right:** It looks like simple substitution, like in algebra. > **The fix:** Densities are mass-*per-unit-length*; changing the variable changes the unit of length, so you MUST multiply by $\left|\frac{dx}{dy}\right|$. Sanity check: your $f_Y$ must integrate to 1. Without the Jacobian it usually won't. > [!mistake] Differentiating the wrong function > **Wrong:** using $\left|\frac{dy}{dx}\right| = |g'(x)|$ directly multiplied in. > **Why it feels right:** $g'$ is the function you were given, so it seems natural. > **The fix:** the formula needs $\left|\frac{dx}{dy}\right| = \frac{1}{|g'(x)|}$, the derivative of the *inverse*. They're reciprocals. Use whichever you can compute, but make sure it's $dx/dy$ in the end. > [!mistake] Dropping a branch in non-monotonic maps > **Wrong:** treating $Y=X^2$ with only $x=+\sqrt y$. > **The fix:** list *all* roots of $g(x)=y$ and sum their contributions. Missing $x=-\sqrt y$ halves your density. > [!mistake] Ignoring the support / range > **Wrong:** writing $f_Y$ for all $y$. > **The fix:** $f_Y$ is nonzero only where $y=g(x)$ for $x$ in the support of $X$. E.g. $Y=X^2\ge 0$, so $f_Y(y)=0$ for $y<0$. --- ## Forecast-then-Verify drill > [!recall] Predict before computing > Let $X\sim\text{Uniform}(0,1)$ and $Y=X^2$. > 1. Will $f_Y$ be larger near $y=0$ or near $y=1$? (Forecast.) > 2. Compute it. > > *Answer:* roots: only $x=+\sqrt y$ (since $X>0$). $\frac{dx}{dy}=\frac{1}{2\sqrt y}$, $f_X=1$. So $f_Y(y)=\frac{1}{2\sqrt y}$ on $(0,1)$. It **blows up near $y=0$** — correct forecast: small $X$ values get compressed near $0$ when squared, piling up density there. --- ## #flashcards/maths What single quantity does the change-of-variable technique conserve? ::: Probability mass, $f_X(x)\,|dx| = f_Y(y)\,|dy|$. State the 1D monotonic change-of-variable formula. ::: $f_Y(y)=f_X(g^{-1}(y))\left|\frac{d}{dy}g^{-1}(y)\right|$. Why is there an absolute value on the Jacobian? ::: Densities must stay nonnegative; for decreasing $g$ the derivative is negative, and the $|\cdot|$ also packages both monotonic cases into one formula. For non-monotonic $g$, how do you build $f_Y$? ::: Sum over all roots $x_i$ of $g(x)=y$: $\sum_i f_X(x_i)\left|dx_i/dy\right|$. For $Y=aX+b$, what is $f_Y$? ::: $f_Y(y)=\frac{1}{|a|}f_X\!\left(\frac{y-b}{a}\right)$. If $X\sim N(0,1)$, what is the distribution of $Y=X^2$? ::: Chi-square with 1 df: $f_Y(y)=\frac{1}{\sqrt{2\pi y}}e^{-y/2}$, $y>0$. How do you generate $\text{Exp}(\lambda)$ from $U\sim\text{Uniform}(0,1)$? ::: $Y=-\frac1\lambda\ln U$; then $f_Y(y)=\lambda e^{-\lambda y}$. In the bivariate case, what replaces the Jacobian factor? ::: The absolute determinant of the Jacobian matrix of the inverse map, $|\det J|$ (the area-scaling factor). Do you use $|dy/dx|$ or $|dx/dy|$ in the final density formula? ::: $|dx/dy|$ — the derivative of the inverse; it's the reciprocal of $|g'(x)|$. Why must you restrict $f_Y$ to the range of $g$? ::: Outside $g(\text{support})$ no probability mass can land, so $f_Y=0$ there. --- > [!recall]- Feynman: explain to a 12-year-old > Imagine you smear a fixed amount of jam (probability) over a ruler. Now you stretch the ruler like rubber: where you stretch it, the jam becomes thinner; where you squeeze it, the jam piles up thicker. The *amount* of jam never changes — only how thick it is at each spot. The "stretch amount" at each point is the Jacobian $|dx/dy|$. The change-of-variable formula just says: new thickness = old thickness × (how much you squeezed there). If your function folds the ruler so two old spots land on the same new spot, you simply add the two layers of jam. > [!mnemonic] > **"Invert, Differentiate, Absolute, Sum, Support"** → **I D A S S** ("I Did A Smart Switch"). > Invert $g$, Differentiate the inverse, take Absolute value, Sum over branches, restrict to the Support. ## Connections - [[Cumulative Distribution Function]] — the safe fallback method (CDF technique) used to *derive* this formula. - [[Jacobian Determinant]] — the multivariable generalization of $|dx/dy|$. - [[Normal Distribution]] — standardization $Z=\frac{X-\mu}{\sigma}$ is a linear transform. - [[Chi-square Distribution]] — arises from $X^2$ of a standard normal. - [[Inverse Transform Sampling]] — uses $X=F^{-1}(U)$, a direct application. - [[Expectation and LOTUS]] — alternative when you only need moments, not the full density. - [[Moment Generating Functions]] — another route to find distributions of transforms/sums. ## 🖼️ Concept Map ```mermaid flowchart TD X[Known dist of X] -->|apply Y = g of X| Q[What is dist of Y?] CONS[Probability mass conserved] -->|core principle| EQ[fY dy = fX dx] Q -->|answered by| EQ EQ -->|via CDF, g monotonic| DER[Differentiate FY] DER -->|chain rule| FORM[Change-of-variable formula] FORM -->|contains| JAC[Jacobian abs dx/dy] JAC -->|is the| STRETCH[Stretch factor of density] FORM -->|g not one-to-one| SUM[Sum over branches] SUM -->|add contributions| PRE[Each preimage xi] FORM -->|example Y = aX+b| LIN[Affine: divide by abs a] ``` ## 🔊 Hinglish (regional understanding) > [!intuition]- Hinglish mein samjho > Dekho, idea bahut simple hai: tumhare paas ek random variable $X$ hai jiska distribution $f_X$ pata hai, aur tum koi function laga dete ho, $Y=g(X)$. Sawal yeh ki $Y$ ka density kya hoga? Yaad rakho — **probability mass conserve hoti hai, density nahi**. Matlab jam ki tarah, $[x,x+dx]$ par jo mass $f_X(x)\,dx$ thi, wahi mass naye interval $[y,y+dy]$ par aa jaati hai. Bas dono ko barabar likho: $f_Y(y)\,|dy| = f_X(x)\,|dx|$. Yahin se sab kuch nikal aata hai. > > Formula yeh ban jaata hai: $f_Y(y)=f_X(g^{-1}(y))\,|dx/dy|$. Woh $|dx/dy|$ wala factor **Jacobian** hai — yeh batata hai ki axis kitni stretch ya squeeze hui. Agar tum stretch karoge to density patli ho jaayegi, squeeze karoge to mote ho jaayegi. Steps yaad rakhne ke liye: **Invert** karo ($x$ ko $y$ se likho), **Differentiate** karo inverse ko, **Absolute** value lo, agar function one-to-one nahi hai to saare roots ka **Sum** lo, aur **Support** (range) ka dhyaan rakho. > > Sabse common galti — log Jacobian bhool jaate hain aur sirf $f_X(g^{-1}(y))$ likh dete hain. Galat! Tab tumhara density integrate karke 1 nahi aayega. Doosri galti: $Y=X^2$ jaise non-monotonic case mein sirf $+\sqrt y$ lete hain, $-\sqrt y$ bhool jaate hain — dono branches ka mass add karna zaroori hai (isi liye normal ka square chi-square ban jaata hai). > > Yeh technique kyun important hai? Real life mein har jagah variables transform hote hain — standardize karna ($Z=\frac{X-\mu}{\sigma}$), energy nikalna ($X^2$), aur computer mein random numbers generate karna (uniform se exponential: $Y=-\frac1\lambda\ln U$). Ek baar jam wali picture dimaag mein baith gayi, to tum kabhi galti nahi karoge. ![[audio/4.9.14-Transformations-of-random-variables-—-change-of-variable-technique.mp3]]