ELBO objective and KL term

4.5.5 · AI-ML › Generative Models

Overview

Evidence Lower BOund (ELBO) variational inference mein central optimization objective hai aur Variational Autoencoders (VAEs) ka training loss bhi. Yeh ek tractable tarika deta hai intractable posterior distributions ko approximate karne ka — log-likelihood par ek lower bound maximize karke — aur saath hi ek KL divergence term bhi include karta hai jo approximate posterior ko regularize karta hai.

Derivation from First Principles

Step 1: Intractable log-likelihood se shuru karo

Hum observed data $x$ ke liye $\log p_{\theta }(x)$ maximize karna chahte hain, jahan generative model yeh hai: $p_{\theta }(x)=\int p_{\theta }(x|z)p(z)dz$

Yeh mushkil kyun hai: $p_{\theta }(x|z)$ ke zyaadatar choices ke liye (jaise ek neural network decoder), is integral ka koi closed form nahi hota aur yeh $z$ ki dimension mein exponentially expensive hota hai.

Step 2: Approximate posterior introduce karo

Kyunki hum $p_{\theta }(z|x)$ directly compute nahi kar sakte, hum ek approximate posterior $q_{\varphi }(z|x)$ introduce karte hain (VAEs mein encoder network). Yeh hamara variational distribution hai — hum $\varphi$ optimize karenge taaki $q_{\varphi }(z|x)$, $p_{\theta }(z|x)$ ke paas aa sake.

Step 3: Jensen's inequality use karke ELBO derive karo

Log-likelihood se shuru karo: $\log p_{\theta }(x)=\log \int p_{\theta }(x,z)dz$

$q_{\varphi }(z|x)$ se multiply aur divide karo: $=\log \int \frac{q_{\varphi }(z|x)}{q_{\varphi }(z|x)}{p}_{\theta }(x,z)dz=\log {\mathbb{E}}_{q_{\varphi }(z|x)}\left[\frac{p_{\theta }(x,z)}{q_{\varphi }(z|x)}\right]$

Key insight: $\log (\cdot )$ ek concave function hai, isliye Jensen's inequality se (jo concave functions ke liye $f(\mathbb{E}[X])\ge \mathbb{E}[f(X)]$ deta hai): $\log {\mathbb{E}}_q[f(z)]\ge {\mathbb{E}}_q[\log f(z)]$

Isko apply karte hain: $\log p_{\theta }(x)\ge {\mathbb{E}}_{q_{\varphi }(z|x)}\left[\log \frac{p_{\theta }(x,z)}{q_{\varphi }(z|x)}\right]$

Yeh step kyun? Jensen's inequality humein $\log$ ko expectation ke andar le jaane deti hai, iske badle mein ek inequality introduce hoti hai. Kyunki $\log$ concave hai, inequality sahi direction mein point karti hai ($\log$ of average ≥ average of $\log$s), aur humein ek valid lower bound milta hai. Isse expression Monte Carlo sampling ke zariye computable ho jaata hai.

Step 4: Conditional aur prior use karke rewrite karo

$p_{\theta }(x,z)=p_{\theta }(x|z)p(z)$ use karke: $\mathcal{L}={\mathbb{E}}_{q_{\varphi }(z|x)}[\log p_{\theta }(x|z)+\log p(z)-\log q_{\varphi }(z|x)]$

Rearrange karo: $={\mathbb{E}}_{q_{\varphi }(z|x)}[\log p_{\theta }(x|z)]+{\mathbb{E}}_{q_{\varphi }(z|x)}\left[\log \frac{p(z)}{q_{\varphi }(z|x)}\right]$

Yeh form kyun?

• Pehla term model ko reward karta hai ki encoder se sample kiye gaye latent codes se $x$ ko reconstruct karo.
• Doosra term encoder distribution ko penalize karta hai agar woh prior $p(z)$ se bhatkti hai, taaki woh collapse ya overfit na kare.

Understanding the KL Term

Humein iska zaroorat kyun hai:

1. Posterior collapse rokta hai: Without the KL term, the encoder could learn to map each $x$ to a deterministic, wildly different $z$, making the latent space discontinuous and unusable for generation.
2. Sampling enable karta hai: $q_{\varphi }(z|x)\approx p(z)$ rakhne se hum $z\sim p(z)$ draw karke aur decode karke naye samples generate kar sakte hain, chahe woh $x$ humne kabhi dekha na ho.
3. Regularization: Complexity penalty ki tarah kaam karta hai — simpler, zyaada "prior-like" encodings prefer ki jaati hain.

Mathematical Properties

Key properties:

• $D_{KL}(q\Vert p)\ge 0$ hamesha (Gibbs' inequality)
• $D_{KL}(q\Vert p)=0$ iff $q=p$ almost everywhere
• Symmetric nahi: $D_{KL}(q\Vert p)\ne D_{KL}(p\Vert q)$ generally

Yeh direction kyun? Hum $D_{KL}(q\Vert p)$ use karte hain $D_{KL}(p\Vert q)$ nahi, kyunki:

• $q$ woh distribution hai jise hum control karte hain (encoder)
• Expectations $q$ ke w.r.t. li jaati hain, jisse hum sample kar sakte hain
• Ise reverse KL ya exclusive KL kehte hain — yeh $q$ ko $p$ ke support ko underestimate karvata hai (zero-forcing behavior)

The Gap: ELBO vs True Log-Likelihood

Step 5: Gap quantify karna

Bayes' rule se shuru karo: $p_{\theta }(z|x)=\frac{p_{\theta }(x|z)p(z)}{p_{\theta }(x)}$

Log lena: $\log p_{\theta }(x)=\log p_{\theta }(x|z)+\log p(z)-\log p_{\theta }(z|x)$

$q_{\varphi }(z|x)$ par expectation lo: $\log p_{\theta }(x)={\mathbb{E}}_{q_{\varphi }}[\log p_{\theta }(x|z)]+{\mathbb{E}}_{q_{\varphi }}[\log p(z)]-{\mathbb{E}}_{q_{\varphi }}[\log p_{\theta }(z|x)]$

${\mathbb{E}}_{q_{\varphi }}[\log q_{\varphi }(z|x)]$ add aur subtract karo: $=\underset{\mathcal{L}(\theta ,\varphi ;x)}{\underbrace{{\mathbb{E}}_{q_{\varphi }}[\log p_{\theta }(x|z)]-D_{KL}(q_{\varphi }(z|x)\Vert p(z))}}+\underset{\text{Gap}}{\underbrace{D_{KL}(q_{\varphi }(z|x)\Vert p_{\theta }(z|x))}}$

Yeh kyun matter karta hai:

• ELBO aur true log-likelihood ke beech ka gap exactly humari approximation aur true posterior ke beech ka KL divergence hai
• Jab hum ELBO maximize karte hain, hum simultaneously generative model ($\theta$) improve karte hain aur approximation quality ($\varphi$) bhi
• Jab $q_{\varphi }(z|x)=p_{\theta }(z|x)$, ELBO log-likelihood ke barabar ho jaata hai

Worked Examples

ELBO: $\mathcal{L}={\mathbb{E}}_{z\sim q_{\varphi }}[-\frac{1}{2}\Vert x-{\mu }_{\theta }(z){\Vert }^2]-D_{KL}(q_{\varphi }(z|x)\Vert \mathcal{N}(0,I))$

Yeh step kyun? Reconstruction term negative squared error ban jaata hai kyunki $\log \mathcal{N}(x;\mu ,I)=-\frac{1}{2}\Vert x-\mu {\Vert }^2+\text{const}$.

Diagonal Gaussians ke liye closed-form KL: $D_{KL}(q_{\varphi }\Vert p)=\frac{1}{2}{\sum }_{j=1}^d\left({\mu }_j^2+{\sigma }_j^2-\log {\sigma }_j^2-1\right)$

Is formula ki derivation: Definition se shuru karte hain: $D_{KL}={\mathbb{E}}_q[\log q(z)-\log p(z)]$ $={\mathbb{E}}_q\left[-\frac{1}{2}{\sum }_j\frac{(z_j-{\mu }_j{)}^2}{{\sigma }_j^2}-\frac{1}{2}\log {\sigma }_j^2+\frac{1}{2}{\sum }_j{z}_j^2\right]+\text{const}$

${\mathbb{E}}_q[z_j^2]={\mu }_j^2+{\sigma }_j^2$ aur ${\mathbb{E}}_q[(z_j-{\mu }_j{)}^2]={\sigma }_j^2$ use karke: $=\frac{1}{2}{\sum }_j\left(-1-\log {\sigma }_j^2+{\mu }_j^2+{\sigma }_j^2\right)$

Yeh kyun matter karta hai: Yeh closed form VAE training ko efficient banata hai — KL term ke liye koi Monte Carlo estimation ki zaroorat nahi.

Step 1: Reconstruction term $\log p_{\theta }(x|z)=-\frac{1}{2}\Vert (0.2,0.8)-(0.19,0.79){\Vert }^2=-\frac{1}{2}(0.01^2+0.01^2)=-0.0001$

Yeh step kyun? Hum evaluate karte hain ki decoder ne sampled latent code se original $x$ kitni acchi tarah reconstruct kiya.

Step 2: KL term ($-\log {\sigma }_j^2=-2\log {\sigma }_j$ use karke, toh $-\log (0.8^2)\approx 0.446$ aur $-\log (1.2^2)\approx -0.365$) $D_{KL}=\frac{1}{2}[(0.5^2+0.8^2-\log (0.8^2)-1)+((-0.3{)}^2+1.2^2-\log (1.2^2)-1)]$ $=\frac{1}{2}[(0.25+0.64+0.446-1)+(0.09+1.44-0.365-1)]$ $=\frac{1}{2}[0.336+0.165]=0.2505$

ELBO: $\mathcal{L}=-0.0001-0.2505=-0.2506$

Yeh step kyun? Hum encoder ko penalize karte hain kyunki usne ek aisi distribution produce ki jo standard normal prior se bhatkti hai. Ek important detail yaad raho: KL formula $\log {\sigma }_j^2=2\log {\sigma }_j$ use karta hai, na ki $\log {\sigma }_j$ — 2 ka factor bhool jaana ek common galti hai.

Kya hota hai:

• $\beta <1$: Reconstruction ko priority deta hai, KL bada ho sakta hai → behtar reconstructions, entangled latents
• $\beta =1$: Standard ELBO, balanced trade-off
• $\beta >1$: Prior se match karne ko priority deta hai → zyaada disentangled latents, worse reconstructions

Yeh kyun kaam karta hai: KL term ki strength information bottleneck control karti hai. Zyaada $\beta$ encoder ko majboor karta hai ki $z$ ke kam, zyaada independent dimensions use kare, jisse disentanglement hoti hai.

Common Mistakes and Misconceptions

Sach yeh hai: KL term ad-hoc regularization nahi hai; yeh variational inference framework se naturally aati hai. Yeh tumhare approximate posterior aur true posterior ke beech ka gap hai. Ise remove karna tumhe "kam regularized" VAE nahi deta — yeh mathematical justification tod deta hai ki tum yeh objective optimize kyun kar rahe ho.

Kaise theek karo: Derivation yaad raho: ELBO + KL gap = log-likelihood. KL term bound ka hissa hai, add-on nahi.

Nuance yeh hai: $\theta$ ke w.r.t. ELBO maximize karna $\log p_{\theta }(x)$ zaroor improve karta hai. Lekin jab hum $\varphi$ bhi optimize karte hain, hum KL gap reduce karke bound tighten kar rahe hote hain. Log-likelihood tabhi badhti hai jab ELBO ki badhot gap reduction se zyaada ho. Practice mein hum jointly optimize karte hain, isliye dono saath hota hai.

Kaise theek karo: ELBO fixed $\varphi$ ke liye lower bound hai. Jab $\varphi$ change hota hai, bound khud move karta hai. Guarantee yeh hai: $\mathcal{L}\le \log p(x)$ hamesha, lekin $\mathcal{L}$ improve karna model improve karta hai.

Sach yeh hai: KL asymmetric hai. $D_{KL}(q\Vert p)$ (reverse KL, ELBO mein use hota hai) zero-forcing hai: yeh $q(z)\approx 0$ banata hai jahan $p(z)\approx 0$ ho, jisse true posterior ke support ka underestimation hota hai. $D_{KL}(p\Vert q)$ (forward KL) zero-avoiding hai: yeh $q(z)>0$ banata hai jahan $p(z)>0$ ho, jisse support ka overestimation hota hai.

VAEs ke liye kyun matter karta hai: Reverse KL posterior collapse cause kar sakta hai jahan encoder true posterior ke kuch modes ignore kar deta hai. Forward KL approximate posterior ko overly broad aur diffuse bana deta.

Kaise theek karo: Direction yaad raho: humein $q$ ke w.r.t. expectations chahiye (jo hum sample kar sakte hain), isliye hum $D_{KL}(q\Vert p)$ use karte hain.

Sach yeh hai: ELBO log-probability par lower bound hai, jo negative ho sakti hai (probabilities 1 se kam ho sakti hain). Jo matter karta hai woh yeh hai ki training ke dauran ELBO badhta rahe, uski absolute value nahi. Ek typical ELBO value data dimensionality ke hisaab se -1000 ya -10000 bhi ho sakti hai.

Kaise theek karo: ELBO improvement track karo, absolute value nahi. Meaningful comparisons ke liye same data par different architectures ka ELBO compare karo.

Active Recall Practice

Toh tum ek guesser machine banate ho jo picture dekh kar guess karta hai ki use kaunse secret code ne banaya. Ab ek game khelo:

1. Guesser ko picture dikhao → woh ek code guess karta hai
2. Woh code magic box mein daalo → dekho ki wahi picture wapas milti hai ya nahi
3. Agar picture sahi nikli toh points milte hain

Lekin ek pakad hai! Tumhara guesser koi bhi random codes nahi bana sakta — use waise codes use karne hote hain jo magic box naturally use karta hai (jaise 0 aur 1 ke beech ke numbers, 999 jaisi pagal numbers nahi).

ELBO tumhara is game mein score hai: Pictures sahi se recreate karne par points milte hain (reconstruction term), lekin agar tumhara guesser weird, unnatural codes use kare toh points katte hain (KL term). ELBO maximize karna matlab dono mein behtar hona — codes guess karna aur pictures recreate karna — aur saath mein apne guesses ko reasonable rakhna.

KL direction ke liye: "Queen Prefers Priors" → $D_{KL}({\mathbf{Q}}_{\varphi }\Vert \mathbf{P})$, hum measure karte hain ki hamari Queen (encoder) Prior se kitni door hai.

Connections

• Variational Inference: ELBO, VI mein fundamental objective hai; VAEs iska deep learning application hain
• KL Divergence: Properties, forward vs reverse, f-divergences mein deep dive
• VAE Architecture: ELBO ko reparameterization trick ke saath neural networks mein kaise implement karte hain
• Expectation Maximization: Latent variable models ke liye EM algorithm mein bhi ELBO aata hai
• Information Bottleneck: β-VAE, weighted KL term ke zariye IB principle se connect hota hai
• Posterior Collapse: Kya hota hai jab KL → 0 aur model latent variables ignore kar deta hai
• Importance Weighted Autoencoders: Multiple samples use karke ELBO se tighter bounds
• Normalizing Flows: Zyaada expressive $q_{\varphi }$ ke liye diagonal Gaussian encoder ka alternative

Summary

ELBO objective $\mathcal{L}={\mathbb{E}}_{q_{\varphi }(z|x)}[\log p_{\theta }(x|z)]-D_{KL}(q_{\varphi }(z|x)\Vert p(z))$ intractable log-likelihood $\log p_{\theta }(x)$ par ek tractable lower bound hai. Yeh do parts mein decompose hota hai:

1. Reconstruction term: Accurate decoding ko reward karo
2. KL regularization: Encoder ko prior ke paas rakho

ELBO aur log-likelihood ke beech ka gap exactly $D_{KL}(q_{\varphi }(z|x)\Vert p_{\theta }(z|x))$ hai — variational posterior ka approximation error. ELBO maximize karna jointly generative model improve karta hai aur approximation gap reduce karta hai. KL term ad-hoc regularization nahi hai balki variational framework ka fundamental consequence hai jo sampling enable karta hai, collapse rokta hai, aur VAEs ke liye ek principled training objective provide karta hai.

#flashcards/ai-ml

Concept Map