Reparameterization trick — Hinglish notes

4.5.4 · AI-ML › Generative Models

Core Problem

VAE mein, encoder ek distribution $q_{ϕ} (z ∣ x)$ ke parameters output karta hai, typically $N (μ_{ϕ} (x), σ_{ϕ}^{2} (x))$ . Hume yeh karna hai:

Sample $z \sim q_{ϕ} (z ∣ x)$
$z$ involve karte hue loss compute karo
$ϕ$ (encoder parameters) update karne ke liye backpropagate karo

Reparameterization Trick: Scratch se Derivation

Step-by-Step Construction

Gaussian distributions ki property se shuru karo:

Agar $ϵ \sim N (0, 1)$ (standard normal) hai, toh kisi bhi $μ$ aur $σ > 0$ ke liye:

$z = μ + σ \cdot ϵ \sim N (μ, σ^{2})$

Yeh step kyun? Yeh Gaussian ki location-scale property hai. Hum ek standard normal ko shift ( $μ$ ) aur stretch ( $σ$ ) kar rahe hain.

Proof:

$E [z] = E [μ + σ ϵ] = μ + σ \cdot 0 = μ$ ✓
$Var (z) = Var (μ + σ ϵ) = σ^{2} \cdot Var (ϵ) = σ^{2} \cdot 1 = σ^{2}$ ✓

Ab critical rewrite:

Yeh likhne ki jagah: $z \sim q_{ϕ} (z ∣ x) = N (μ_{ϕ} (x), σ_{ϕ}^{2} (x))$

Hum likhte hain: $ϵ \sim N (0, 1)$ $z = μ_{ϕ} (x) + σ_{ϕ} (x) \cdot ϵ$

Yeh step kyun? Ab $z$ ek deterministic function hai $ϕ$ ka (through $μ_{ϕ}$ aur $σ_{ϕ}$ ) aur random noise $ϵ$ ka. Randomness sirf $ϵ$ mein rehti hai, jo hamare parameters par depend nahi karti.

Gradient Flow

Loss typically aisi dikhti hai: $L (ϕ) = E_{q_{ϕ} (z ∣ x)} [f (z, x)]$

Purana tarika (kaam nahi karta): $\nabla_{ϕ} L = E_{q_{ϕ} (z ∣ x)} [\nabla_{ϕ} f (z, x)] \leftarrow STUCK! Can’t compute \nabla_{ϕ} z$

Naya tarika (reparameterized): $L (ϕ) = E_{p (ϵ)} [f (g_{ϕ} (ϵ, x), x)]$

$\nabla_{ϕ} L = E_{p (ϵ)} [\nabla_{ϕ} f (g_{ϕ} (ϵ, x), x)]$

Yeh step kyun? Expectation ab $p (ϵ)$ par hai, jo $ϕ$ par depend nahi karti. Hum gradient aur expectation ko swap kar sakte hain:

$= E_{p (ϵ)} [\frac{\partial f}{\partial z} \cdot \frac{\partial g _{ϕ}}{\partial ϕ}]$

Chain rule kaam karta hai! $\frac{\partial g _{ϕ}}{\partial ϕ}$ mein simple derivatives hain:

$\frac{\partial}{\partial μ _{ϕ}} (μ_{ϕ} + σ_{ϕ} ϵ) = 1$
$\frac{\partial}{\partial σ _{ϕ}} (μ_{ϕ} + σ_{ϕ} ϵ) = ϵ$

Worked example Worked Example 1: Single Sample Gradient

Setup: Encoder $μ = 2.0$ , $σ = .5$ output karta hai. Hum $N (0, 1)$ se $ϵ = 1.3$ sample karte hain। Loss function: $f (z) = (z - 3)^{2}$ ( $z$ ko 3 ke paas chahte hain). Step 1: Reparameterization se $z$ compute karo. $z = μ + σ \cdot ϵ = 2.0 + 0.5 \times 1.3 = 2.65$

Yeh step kyun? Yeh hamare deterministic transform se ek concrete sample deta hai.

Step 2: Loss compute karo. $f (z) = (2.65 - 3)^{2} = 0.1225$

Step 3: $\frac{\partial f}{\partial z}$ compute karo (loss ka latent ke w.r.t. gradient). $\frac{\partial f}{\partial z} = 2 (z - 3) = 2 (2.65 - 3) = - 0.7$

Yeh step kyun? Hume jaanna hai ki $f$ $z$ ke saath kaise change hota hai taaki chain rule apply kar sakein.

Step 4: $\frac{\partial z}{\partial μ}$ compute karo (latent mean ke saath kaise change hota hai). $\frac{\partial z}{\partial μ} = \frac{\partial}{\partial μ} (μ + σ ϵ) = 1$ Yeh step kyun? Reparameterization $z$ ko $μ$ par linearly dependent banata hai.

Step 5: $μ$ ke liye chain rule. $\frac{\partial f}{\partial μ} = \frac{\partial f}{\partial z} \cdot \frac{\partial z}{\partial μ} = - 0.7 \times 1 = - 0.7$

Interpretation: $z$ badhane aur loss kam karne ke liye $μ$ badhao. Gradient descent update karega: $μ \leftarrow 2.0 - lr \times (- 0.7) = 2.0 + 0.7 \times lr$ . ✓

Step 6: $\frac{\partial z}{\partial σ}$ compute karo. $\frac{\partial z}{\partial σ} = \frac{\partial}{\partial σ} (μ + σ ϵ) = ϵ = 1.3$ Yeh step kyun? Noise $ϵ$ $σ$ ke liye sensitivity factor ki tarah kaam karta hai.

Step 7: $σ$ ke liye chain rule. $\frac{\partial f}{\partial σ} = \frac{\partial f}{\partial z} \cdot \frac{\partial z}{\partial σ} = - 0.7 \times 1.3 = - 0.91$

Interpretation: $σ$ badhana (positive $ϵ$ ke saath) $z$ badhata hai, loss kam karta hai. ✓

Worked example Worked Example 2: Monte Carlo Estimation

Setup: Hamare paas analytic expectation nahi hai, toh hum $S$ samples se approximate karte hain.

$\nabla_{ϕ} L = E_{p (ϵ)} [\nabla_{ϕ} f (g_{ϕ} (ϵ, x), x)] \approx \frac{1}{S} \sum_{s = 1}^{S} \nabla_{ϕ} f (g_{ϕ} (ϵ^{(s)}, x), x)$ Step 1: $S$ independent samples $ϵ^{(1)}, \dots, ϵ^{(S)} \sim N (0, 1)$ draw karo.

Yeh step kyun? Monte Carlo integration—hum expectation ko sampling se approximate kar rahe hain.

Step 2: Har sample $s$ ke liye compute karo: $z^{(s)} = μ_{ϕ} (x) + σ_{ϕ} (x) \cdot ϵ^{(s)}$

Step 3: Individual gradients compute karo: $g^{(s)} = \nabla_{ϕ} f (z^{(s)}, x)$

Step 4: Average karo: $\overset{g}{^} = \frac{1}{S} \sum_{s = 1}^{S} g^{(s)}$

Yeh step kyun? Law of large numbers: jaise $S \to \infty$ , $\overset{g}{^} \to E [g]$ . Chota $S$ bhi (jaise 1-10) practice mein neural network losses ki smoothness ki wajah se achha kaam karta hai.

Practical VAE: Aksar $S = 1$ (ek hi sample) per training step! Mini-batches mein stochasticity kaafi gradient signal provide karti hai.

Yeh Kyun Kaam Karta Hai: Math

Proof sketch: Change of variables. Agar $z = μ + σ ϵ$ , toh $d z = σ d ϵ$ , aur: $p_{z} (z) d z = p_{ϵ} (\frac{z - μ}{σ}) \frac{d z}{σ}$

Distributions construction se match karte hain. Expectations equal hain.

Gradient trick isliye kaam karta hai kyunki:

$ϵ$ par expectation $ϕ$ par depend nahi karti
Leibniz integral rule: $\nabla_{ϕ} \int p (ϵ) f (g_{ϕ} (ϵ)) d ϵ = \int p (ϵ) \nabla_{ϕ} f (g_{ϕ} (ϵ)) d ϵ$
Hum gradient ko expectation ke andar push kar sakte hain

Practical Implementation

# PyTorch pseudocode
def reparameterize(mu, log_var):
    """
    mu: [batch, latent_dim] - mean from encoder
    log_var: [batch, latent_dim] - log variance (for numerical stability)
    """
    std = torch.exp(0.5 * log_var)  # sigma = exp(log_var / 2)
    eps = torch.randn_like(std)      # sample epsilon ~ N(0, 1)
    z = mu + std * eps               # reparameterization
    return z

Common Mistakes

Common mistake Mistake 1:

$ϵ$ ko Detach Karna Bhool Jana Galat code:

eps = torch.randn_like(std)
eps.requires_grad = True  # DON'T DO THIS
z = mu + std * eps

Yeh sahi kyun lagta hai: "Neural network mein har cheez ko gradients chahiye, hai na?"

Yeh galat kyun hai: $ϵ$ ek fixed distribution se sample kiya gaya hai. $ϵ$ ke w.r.t. gradients meaningless hain—hum random seed optimize nahi kar rahe! requires_grad=True set karna computation waste karta hai aur bugs cause kar sakta hai.

Fix: $ϵ$ ko as-is rakho. torch.randn_like() by default requires_grad=False wala tensor create karta hai. Gradient automatically mu aur std ke through addition aur multiplication operations se flow karega.

Common mistake Mistake 2:

$σ$ ki Jagah $σ^{2}$ Use Karna Galat:

var = torch.exp(log_var)
z = mu + var * eps  # variance se multiply kar rahe hain, std se nahi!

Yeh sahi kyun lagta hai: Encoder variance output karta hai, toh seedha use karo.

Yeh galat kyun hai: Reparameterization formula $z = μ + σ ϵ$ hai, $z = μ + σ^{2} ϵ$ nahi. Standard deviation ki jagah variance use karna noise ko galat scale karta hai. Agar $σ = 0.5$ , toh $σ^{2} = 0.25$ —aap spread ko aadha underestimate kar rahe hoge!

Fix: Square root lo. Kyunki hum typically log variance se parameterize karte hain (numerical stability ke liye): $σ = exp (\frac{1}{2} lo g σ^{2}) = exp (0.5 \times log_var)$

Generalizations

Yeh trick Gaussians se aage bhi extend hoti hai:

Distribution	Reparameterization
Gaussian	$z = μ + σ ϵ$ , $ϵ \sim N (0, 1)$
Exponential	$z = - \frac{1}{λ} lo g u$ , $u \sim Uniform (0, 1)$
Logistic	$z = μ + s lo g \frac{u}{1 - u}$ , $u \sim Uniform (0, 1)$
Laplace	$z = \mu - b \cdot \text{sgn}(u-0.5) \log(1-2

Key requirement: Distribution ka ek location-scale form hona chahiye ya ek invertible CDF (inverse transform sampling).

Recall Ek 12-Saal-Ke Bachhe Ko Samjhao

Socho aap ek robot ko drawing sikhana chahte ho. Robot ke paas ek "creativity knob" hai (woh hai $σ$ ) aur ek "center point" knob hai (woh hai $μ$ ). Har baar jab woh draw karta hai, woh center se move karne ke liye ek random direction aur distance choose karta hai.

Problem yeh hai: agar drawing zyada wobbly (random) ho, toh robot nahi seekh sakta ki kaun si knob settings achhi art banati hain. Yeh aisa hai jaise basketball shot improve karne ki koshish karo jab koi tumhara arm randomly push kare—tum nahi bata sakte ki woh tumhari aim thi ya push!

Reparameterization trick aisi hai: robot ko randomly direction AUR distance dono choose karne ki jagah (seekhne ke liye zyada random), hum usse ek standard random step dete hain (jaise hamesha ek regular die roll karo), aur phir robot us step ko apni knobs se scale karta hai. Ab agar drawing buri ho, robot jaanta hai ki knob settings ki wajah se tha, randomness ki wajah se nahi.

Knobs ek fixed random step ke transform ko control karte hain. Woh transform smooth aur learnable hai!

Connections

4.5.01-Variational-Autoencoders - primary application
4.5.02-ELBO-Derivation - jahan yeh trick gradient-based optimization enable karti hai
4.3.01-KL-Divergence - ELBO ka KL term Gaussians ke liye closed form mein hai
5.2.03-Gumbel-Softmax - discrete analog
2.4.01-Backpropagation - kyun gradients ko differentiable paths chahiye
6.1.02-Policy-Gradients - REINFORCE (score function method) se contrast

Flashcards

#flashcards/ai-ml

Reparameterization trick kaun si core problem solve karta hai? :: VAEs mein, hume ek stochastic sampling node $z \sim q_{ϕ} (z ∣ x)$ ke through backpropagate karna hota hai, lekin sampling distribution parameters $ϕ$ ke w.r.t. non-differentiable hai. Randomness ke through gradients flow nahi kar sakte.

Gaussian distribution ke liye reparameterization trick kya hai?

z \sim N (μ, σ^{2})

directly sample karne ki jagah, ek fixed distribution se

ϵ \sim N (0, 1)

sample karo, phir

z = μ + σ \cdot ϵ

compute karo. Yeh

z

μ

aur

σ

ka differentiable function bana deta hai.

Reparameterization ke baad hum gradient ko expectation ke andar kyun move kar sakte hain?

Reparameterization ke baad, expectation

p (ϵ)

par hai jo parameters

ϕ

par depend nahi karti:

\nabla_{ϕ} E_{p (ϵ)} [f (g_{ϕ} (ϵ)] = E_{p (ϵ)} [\nabla_{ϕ} f (g_{ϕ} (ϵ))]

. Leibniz integral rule gradient aur integral ko swap karne deta hai jab integration domain parameter-free ho.

Jab

z = μ + σ ϵ

ho toh

\frac{\partial z}{\partial μ}

kya hai?

\frac{\partial z}{\partial μ} = 1

(ek constant). Iska matlab hai ki

μ

mein changes

z

ko same amount se shift karte hain.

Jab

z = μ + σ ϵ

ho toh

\frac{\partial z}{\partial σ}

kya hai?

\frac{\partial z}{\partial σ} = ϵ

. Gradient sampled noise value par depend karta hai. Jab

ϵ

bada hota hai,

σ

z

par zyada influence hota hai.

Practice mein, VAEs reparameterization gradient estimator ke liye kitne samples

S

use karte hain?

Aksar sirf

S = 1

(har training step mein single sample). Kaafi training steps mein mini-batch stochasticity single-sample estimator ke variance ke bawajood sufficient gradient signal deti hai.

Hum typically variance ki jagah log-variance se kyun parameterize karte hain?

Numerical stability. Variance positive hona chahiye, lekin log-variance koi bhi real number ho sakta hai.

σ = exp (0.5 \cdot log_var)

compute karna ensure karta hai ki

σ > 0

hai aur optimization constraints se bachata hai.

Log-variance se

σ

get karne ka sahi formula kya hai?

σ = exp (0.5 \cdot lo g σ^{2}) = exp (\frac{1}{2} log_var)

, kyunki

lo g σ = \frac{1}{2} lo g σ^{2}

Kya reparameterization trick Categorical jaisi discrete distributions par apply ki ja sakti hai?

Directly nahi. Discrete sampling ka koi gradient nahi hota (argmax non-differentiable hai). Alternatives mein Gumbel-Softmax (continuous relaxation) ya REINFORCE (score function estimator) shamil hain.

Diagonal covariance wale multivariate Gaussian ke liye reparameterization kaise kaam karta hai?

Element-wise apply karo:

ϵ \sim N (0, I_{d})

aur

z = μ + σ ⊙ ϵ

jahan

⊙

element-wise multiplication hai. Har dimension independent hai.