Classifier-free guidance

4.5.14 · AI-ML › Generative Models

Ye Kaunsi Problem Solve Karta Hai?

Conditional diffusion models (jaise text-to-image systems) aksar aisi samples produce karte hain jo high-quality hoti hain lekin conditioning signal (text, class label, etc.) ko strongly follow nahi karti. Early approaches ek separate classifier gradient use karti thi generation ko steer karne ke liye, lekin iske liye ek extra noise-robust classifier train karna padta tha.

Classifier-free guidance classifier ko bilkul khatam kar deta hai ek single model train karke jo dono conditional aur unconditional generation handle kare, aur phir sampling time par unke predictions ko combine kare.

Mathematical Framework

First Principles Se Derivation

STEP 1: Hum kya achieve karna chahte hain?

Hum $p(\mathbf{x}|\mathbf{c})$ se sample karna chahte hain lekin use conditioning $\mathbf{c}$ ko zyada strongly reflect karna chahte hain. Classifier guidance mein, humne score ko ${\nabla }_{{\mathbf{x}}_t}\log p(\mathbf{c}|{\mathbf{x}}_t)$ se modify kiya tha. Yahan hum bina classifier ke equivalent effect derive karenge.

STEP 2: Conditional score ko rewrite karo

Bayes' rule se: $\log p({\mathbf{x}}_t|\mathbf{c})=\log p({\mathbf{x}}_t)+\log p(\mathbf{c}|{\mathbf{x}}_t)-\log p(\mathbf{c})$

Gradient lete hain: ${\nabla }_{{\mathbf{x}}_t}\log p({\mathbf{x}}_t|\mathbf{c})={\nabla }_{{\mathbf{x}}_t}\log p({\mathbf{x}}_t)+{\nabla }_{{\mathbf{x}}_t}\log p(\mathbf{c}|{\mathbf{x}}_t)$

YE STEP KYUN? Last term $\log p(\mathbf{c})$ ${\mathbf{x}}_t$ par depend nahi karta, isliye gradient ke neeche vanish ho jaata hai. Ye conditional score ko unconditional score + classifier gradient mein split karta hai.

STEP 3: Conditioning signal ko amplify karo

Sample ko $\mathbf{c}$ zyada strongly follow karaane ke liye, hum classifier term ko factor $w$ se scale up karte hain: ${\nabla }_{{\mathbf{x}}_t}\log \tilde{p}({\mathbf{x}}_t|\mathbf{c})={\nabla }_{{\mathbf{x}}_t}\log p({\mathbf{x}}_t)+w\cdot {\nabla }_{{\mathbf{x}}_t}\log p(\mathbf{c}|{\mathbf{x}}_t)$

KYUN? Jab $w=1$, hume true conditional milta hai. Jab $w>1$, hum conditioning ki taraf "overshoot" karte hain, jisse outputs $\mathbf{c}$ ke liye zyada specific hote hain diversity ki cost par.

STEP 4: Model ki predictions ke terms mein express karo

Bayes gradient ko rearrange karo: ${\nabla }_{{\mathbf{x}}_t}\log p(\mathbf{c}|{\mathbf{x}}_t)={\nabla }_{{\mathbf{x}}_t}\log p({\mathbf{x}}_t|\mathbf{c})-{\nabla }_{{\mathbf{x}}_t}\log p({\mathbf{x}}_t)$

Guided score mein substitute karo: ${\nabla }_{{\mathbf{x}}_t}\log \tilde{p}({\mathbf{x}}_t|\mathbf{c})={\nabla }_{{\mathbf{x}}_t}\log p({\mathbf{x}}_t)+w\cdot [{\nabla }_{{\mathbf{x}}_t}\log p({\mathbf{x}}_t|\mathbf{c})-{\nabla }_{{\mathbf{x}}_t}\log p({\mathbf{x}}_t)]$

$=(1-w){\nabla }_{{\mathbf{x}}_t}\log p({\mathbf{x}}_t)+w{\nabla }_{{\mathbf{x}}_t}\log p({\mathbf{x}}_t|\mathbf{c})$

STEP 5: Noise prediction mein convert karo

Diffusion models mein, score predicted noise se relate hota hai: ${\nabla }_{{\mathbf{x}}_t}\log p({\mathbf{x}}_t)\propto -{ϵ}_{\theta }({\mathbf{x}}_t,t)$. Toh:

$\boxed{{\widetilde{ϵ}}_{\theta }=(1-w){ϵ}_{\theta }({\mathbf{x}}_t,t,\varnothing )+w{ϵ}_{\theta }({\mathbf{x}}_t,t,\mathbf{c})}$

YE FORM KYUN? Ye ek linear interpolation aur extrapolation hai. Jab $w=0$, purely unconditional. Jab $w=1$, normal conditional. Jab $w>1$, hum conditional se aage "$\mathbf{c}$ se zyada" ki taraf extrapolate karte hain.

Training Procedure

Classifier-free guidance enable karne ke liye, hum ek single model train karte hain jo dono conditional aur unconditional denoising handle kare:

1. Training ke dauran, conditioning $\mathbf{c}$ ko randomly probability $p_{\text{uncond}}$ (typically 10-20%) se drop karo.

   • Jab drop ho, $\mathbf{c}$ ko ek null token $\varnothing$ se replace karo (text ke liye empty string, zero embedding, etc.).

2. Loss same rehta hai: $\mathcal{L}={\mathbb{E}}_{t,{\mathbf{x}}_0,ϵ,\mathbf{c}}\left[\Vert ϵ-{ϵ}_{\theta }({\mathbf{x}}_t,t,\mathbf{c}){\Vert }^2\right]$ jahan $\mathbf{c}$ kabhi kabhi $\varnothing$ hota hai.

YE KYUN KAAM KARTA HAI: Model same parameters mein conditioning ke saath aur bina conditioning ke dono denoise karna seekhta hai. Inference par, hum ise do baar call kar sakte hain (ek $\mathbf{c}$ ke saath, ek $\varnothing$ ke saath) dono predictions paane ke liye.

Worked Examples

Common Mistakes

Doosre Concepts Se Connections

• Score-based diffusion models: CFG score function ko conditional aur unconditional scores combine karke manipulate karta hai.
• Classifier guidance: Woh predecessor approach jo ek separate classifier $p(\mathbf{c}|{\mathbf{x}}_t)$ use karti thi; CFG bina classifier ke same effect derive karta hai.
• DDPM sampling: CFG DDPM reverse process mein use hone wale noise prediction ${ϵ}_{\theta }$ ko modify karta hai.
• Conditional generation: CFG conditional generative models mein conditioning signals ko strengthen karne ki ek technique hai.
• Prompt engineering: Zyada guidance scales prompt wording ko zyada critical banate hain (text par overfitting).
• Negative prompting: Aksar CFG ke saath combine hota hai unwanted features se door push karne ke liye.

CFG Kab Use Karein

Classifier-free guidance tab use karo jab:

• Tum text/class conditioning mein strong adherence chahte ho bina separate classifier train kiye
• Tum text-to-image, text-to-video, ya class-conditional models bana rahe ho
• Tumhe prompt strength ke liye ek tunable knob chahiye

Tab avoid karo ya tune down karo jab:

• Tum maximum sample diversity chahte ho (high $w$ mode collapse cause karta hai)
• Compute cost critical hai (har step par 2× model evaluations chahiye)
• Tum unconditional samples generate kar rahe ho (sirf $w=0$ use karo ya CFG skip karo)

#flashcards/ai-ml

Concept Map