Classifier-free guidance: Question bank — AI-ML deep dive

This bank hunts the conceptual misconceptions around classifier-free guidance (CFG). Every reveal below forces you to justify — a bare "true" or "false" earns nothing. Cover the sign traps, the $w$ -limits, the training-time null token, and the difference from Classifier guidance. Read the question, commit to an answer out loud, then reveal.

Definition The two symbols this whole page rests on

$ϵ_{θ} (x_{t}, t, c)$ is the network's predicted noise: given a noisy image $x_{t}$ at diffusion timestep $t$ and a conditioning input $c$ , the model outputs its best guess of the noise that was added. Removing that predicted noise is what denoises one step. We abbreviate $ϵ_{c} = ϵ_{θ} (x_{t}, t, c)$ and $ϵ_{\emptyset} = ϵ_{θ} (x_{t}, t, \emptyset)$ .
What "score" means here (plain words first): the score of a probability distribution is the gradient $\nabla_{x_{t}} lo g p_{t} (x_{t})$ — an arrow at each point of image-space pointing in the direction that increases the probability of that noisy image fastest. Think of $p_{t}$ as the distribution of images at noise level $t$ (the noised images the model actually sees at that step), not the clean-image distribution.
Its link to the predicted noise (with the constant made explicit): for the standard variance-preserving forward process, if $\overset{α}{ˉ}_{t}$ is the fraction of the original signal surviving to step $t$ , the score and predicted noise are related by $\nabla_{x_{t}} lo g p_{t} (x_{t}) = - \frac{1}{1 - α ˉ _{t}} ϵ_{θ} (x_{t}, t),$ so the proportionality constant is $- 1/ 1 - \overset{α}{ˉ}_{t}$ (a fixed positive scale at each $t$ , not a free knob). The minus sign says: predicted noise points away from clean, the score points toward more-probable. Because the constant is the same for the conditional and unconditional passes, it cancels in every difference below — which is why we can reason entirely in $ϵ$ .

Figure 1 plots the guidance line through the two anchor predictions $ϵ_{\emptyset}$ (point ①, at $w = 0$ ) and $ϵ_{c}$ (point ②, at $w = 1$ ). The cyan segment ③ is the interpolation region $w \in [0, 1]$ where the guided point sits between the anchors; the amber ray ④ is the extrapolation region $w > 1$ where it shoots past $ϵ_{c}$ (the amber star marks $w = 2$ ). Every "sign trap" below is really asking: where on this line does your $w$ land — cyan segment ③ or amber ray ④?

True or false — justify

CFG requires training a separate noise-robust classifier alongside the diffusion model.

False. That is Classifier guidance. CFG's whole point is to avoid the extra classifier by training one model to do both conditional and unconditional denoising, then differencing the two at sampling time.

Setting

w = 1

recovers ordinary conditional sampling with no amplification.

True. At

w = 1

the formula collapses to

(1 - 1) ϵ_{\emptyset} + 1 \cdot ϵ_{c} = ϵ_{c}

, exactly the plain conditional prediction — no guidance, no extrapolation.

w = 0

the model still weakly follows the prompt.

False. At

w = 0

the coefficient on

ϵ_{c}

is zero and on

ϵ_{\emptyset}

is one, so you get the purely unconditional prediction — the prompt is completely ignored.

Increasing

w

always improves image quality.

False. Higher

w

improves prompt adherence but eventually over-saturates colors, flattens texture, and reduces diversity (mode-collapse-like behaviour). Quality peaks at a moderate

w

then degrades.

For

w > 1

, the unconditional term carries a negative weight.

True. The weight is

(1 - w)

, which is negative once

w > 1

(e.g. at

w = 2

it is

- 1

). This actively pushes the sample away from the generic unconditional distribution, sharpening it toward

c

CFG changes what the network's weights are; it is a training-time modification.

False. The trained network is untouched. CFG is an inference-time recombination of two forward passes. The only training-side change is randomly dropping

c

so the same weights can denoise unconditionally.

The vector

Δ ϵ = ϵ_{c} - ϵ_{\emptyset}

is the same length regardless of the prompt.

False.

Δ ϵ

is the "direction the prompt adds," and its magnitude depends on how much the prompt changes the prediction at that

x_{t}, t

. A vague or off-distribution prompt yields a small

Δ ϵ

; a strong, in-distribution prompt yields a large one.

Because CFG runs the model twice per step, it roughly doubles sampling compute versus plain conditional sampling.

True. Each denoising step needs both

ϵ_{c}

and

ϵ_{\emptyset}

, so wall-clock cost per step is about

2 \times

(often batched into one forward pass of doubled batch size).

CFG and Classifier guidance produce the score modification via the same term.

True (in effect). Both add

w \cdot \nabla_{x_{t}} lo g p (c ∣ x_{t})

. CFG just estimates that gradient implicitly as

ϵ_{c} - ϵ_{\emptyset}

instead of reading it off an explicit classifier.

The null token

\emptyset

must be a special learned symbol distinct from any real prompt.

Mostly true in practice. For text it is typically the empty string / a fixed padding embedding; for class labels a dedicated null class. What matters is that it is a consistent, reserved input the model was trained to see when conditioning is dropped.

Spot the error

"CFG works by adding the classifier gradient

\nabla_{x_{t}} lo g p (c)

to the score."

Error: it should be

\nabla_{x_{t}} lo g p (c ∣ x_{t})

. The term

lo g p (c)

has no

x_{t}

dependence, so its gradient is zero — it can't steer anything.

"The guided formula is

\tilde{ϵ} = w ϵ_{\emptyset} + (1 - w) ϵ_{c}

Error: the weights are swapped. It is

(1 - w) ϵ_{\emptyset} + w ϵ_{c}

. As written, at

w > 1

you'd amplify the unconditional term — the opposite of guidance.

"Since

\tilde{ϵ} = ϵ_{\emptyset} + w Δ ϵ

, choosing

w = 0

gives the conditional prediction."

Error:

w = 0

kills

Δ ϵ

, leaving

ϵ_{\emptyset}

— the unconditional prediction. You need

w = 1

for the plain conditional.

"To train CFG we drop the conditioning on 100% of examples so the model becomes fully unconditional."

Error: the drop rate, call it

p_{drop}

, is a small fraction (commonly

p_{drop} \in [0.1, 0.2]

; the original CFG paper found

10 - 20%

worked well, and large values like

p_{drop} = 0.5

waste capacity the conditional path needs). Setting

p_{drop} = 1

would make the model never learn conditional denoising, so

ϵ_{c}

would be meaningless.

Δ ϵ

points from the conditional toward the unconditional prediction."

Error:

Δ ϵ = ϵ_{c} - ϵ_{\emptyset}

points from the unconditional toward the conditional. We then add

w

copies of it to the unconditional baseline to overshoot toward the prompt.

"Because guidance sharpens toward the prompt, CFG increases the diversity of generated samples."

Error: it reduces diversity. Concentrating probability mass around prompt-matching modes is exactly why high

w

causes reduced variety and mode-collapse-like output.

"CFG requires the unconditional and conditional models to be two separately trained networks."

Error: it is one network with shared weights; the "two models" are two forward passes of the same weights, differing only in whether

c

\emptyset

is fed in.

Why questions

Why does the

lo g p (c)

term disappear when we take

\nabla_{x_{t}}

Because

p (c)

is a constant with respect to

x_{t}

(the prior over conditions doesn't depend on the noisy image), and the gradient of a constant is zero. This is what cleanly splits the conditional score into unconditional score plus the classifier term.

Why does subtracting

ϵ_{\emptyset}

from

ϵ_{c}

isolate the effect of the prompt?

Both predictions share everything the model knows about generic images at

(x_{t}, t)

; only the conditional one also encodes

c

. Subtracting cancels the common part and leaves just the prompt-specific direction

Δ ϵ

Why does scaling

Δ ϵ

w

correspond to sampling from the sharpened posterior

p (x_{t}) p (c ∣ x_{t})^{w}

Take the log of that target:

lo g p (x_{t}) + w lo g p (c ∣ x_{t})

. Its gradient is

\nabla lo g p (x_{t}) + w \nabla lo g p (c ∣ x_{t})

. Since the classifier gradient equals

\nabla lo g p (x_{t} ∣ c) - \nabla lo g p (x_{t})

, which in

ϵ

-land is exactly

- (ϵ_{c} - ϵ_{\emptyset}) / 1 - \overset{α}{ˉ}_{t}

, scaling it by

w

is raising

p (c ∣ x_{t})

to the power

w

— a peakier, more confident posterior.

Why must we train with conditioning dropout rather than just feeding

\emptyset

at inference?

If the model never saw

\emptyset

during training, its "unconditional" pass would be an untrained, arbitrary output. Dropout teaches the same weights to give a valid unconditional prediction, which is what

ϵ_{\emptyset}

must be for the difference to be meaningful.

Why does negative prompting fit naturally into the CFG formula?

Negative prompting swaps the

\emptyset

baseline for a prediction conditioned on the unwanted concept

c_{neg}

; guidance then pushes away from

c_{neg}

and toward the positive prompt — the same subtract-and-extrapolate structure with a non-empty baseline.

Why is CFG described as extrapolation rather than interpolation when

w > 1

For

w \in [0, 1]

the guided prediction is a convex blend between

ϵ_{\emptyset}

and

ϵ_{c}

(interpolation, cyan segment ③ in Figure 1). For

w > 1

the point lies beyond

ϵ_{c}

on the line through both (amber ray ④) — you leave the segment, hence extrapolation.

Why is CFG describable in the language of Score-based diffusion models at all?

Because the predicted noise is a fixed rescaling of the negative score,

\nabla_{x_{t}} lo g p_{t} (x_{t}) = - ϵ_{θ} / 1 - \overset{α}{ˉ}_{t}

. So manipulating scores by Bayes' rule translates one-to-one into manipulating noise predictions — that exact relation is what makes the whole derivation legal.

Edge cases

What does

\tilde{ϵ}

equal in the limit

w \to \infty

(direction only)?

It aligns with the ray

ϵ_{\emptyset} + w Δ ϵ

, dominated entirely by

Δ ϵ

. In practice this produces extreme, over-saturated, low-diversity samples long before the theoretical limit.

What happens if a prompt is so out-of-distribution that

Δ ϵ \approx 0

Guidance has almost nothing to amplify, so

\tilde{ϵ} \approx ϵ_{\emptyset}

regardless of

w

— the output stays generic. The prompt effectively fails to steer generation.

w

is set negative, what would CFG do?

The weight on

ϵ_{c}

becomes negative, so sampling is pushed away from the prompt — you'd get an image avoiding the described concept, the inverse of guidance (and the mechanism behind treating an unwanted concept as a target to flee).

At the final denoising step (

t \to 0

, image nearly clean), does the guidance term matter?

It still applies, but

Δ ϵ

is typically small there because the image is already mostly determined; most of guidance's shaping happens at higher

t

where structure is still forming.

Is CFG applicable to unconditional-only models (no

c

ever)?

No. Without any conditioning signal there is no

ϵ_{c}

to compare against

ϵ_{\emptyset}

, so

Δ ϵ

is undefined. CFG fundamentally needs a Conditional generation setup.

Does the DDPM/DDIM update change under CFG?

No — the sampler equations are identical. You simply feed

\tilde{ϵ}

in place of

ϵ_{θ}

wherever the DDPM sampling step uses the predicted noise.

Recall Quick self-check

"Load-bearing sign fact" means the one sign that, if flipped, breaks everything. Here it is ::: For $w > 1$ , the unconditional term's weight $(1 - w)$ is negative, pushing away from generic samples toward the prompt. Flip that sign and you'd steer away from the prompt instead. The single load-bearing training fact ::: Randomly drop $c$ with probability $p_{drop}$ (a small value, commonly $p_{drop} \in [0.1, 0.2]$ ) and replace with $\emptyset$ so one set of weights learns both conditional and unconditional denoising.