4.5.4 · D5Generative Models
Question bank — Reparameterization trick
Before we start, one shared picture. Imagine a knob (the encoder's parameters) and a dice cup (fixed randomness). The whole point of the trick is to route every random effect through the dice cup so that the knob only ever appears in a plain, differentiable formula . The figure below fixes that knob-versus-cup split — keep it in mind for every item.
Once the randomness lives entirely in the cup, the derivative can walk backward along an unbroken path from the loss to the knob. That flow is the second picture:
True or false — justify
The naive sampler has a well-defined derivative .
False. Direct sampling is a stochastic node — nudging changes the distribution, not the specific drawn value, so is undefined as an ordinary derivative and 2.4.01-Backpropagation stalls there.
After reparameterization, is a deterministic function of , and .
True. Given a fixed draw , the map is an ordinary differentiable function; all remaining randomness sits inside , which the gradient never touches.
The reparameterized loss and the original loss are different objectives.
False. They are the same expectation, just rewritten: by change of variables. Only the sampling recipe changed, not what we optimize.
The trick removes randomness from training.
False. It relocates randomness into the parameter-free . Each step is still a noisy Monte-Carlo estimate; the trick only makes that noisy estimate differentiable in .
Reparameterization works for any distribution the encoder might output.
False. It needs a differentiable transform from a fixed base variable to the target. Gaussians (location–scale) work; discrete/categorical latents have no such continuous map — that is exactly why 5.2.03-Gumbel-Softmax exists.
Using sample per step gives a biased gradient estimate.
False. A single-sample estimate is unbiased (its expectation equals the true gradient); it is merely high-variance. Mini-batch averaging supplies enough signal in practice.
The gradient is the same for every training example in a batch.
False. It equals the drawn for that example, so it varies sample-to-sample. That per-sample is precisely the sensitivity factor for .
The reparameterization trick and the score-function (REINFORCE) estimator compute the same thing.
True in expectation, false in practice. Both estimate , but the pathwise estimator here (also called the reparameterization estimator, because the gradient flows along the deterministic path ) typically has far lower variance than the score-function/6.1.02-Policy-Gradients estimator because it uses directly.
Swapping and the expectation is always legal.
False in general, true here. It is licensed only when the sampling density does not depend on (see the Leibniz rule note below). Reparameterization manufactures that condition by pushing out of and into .
Spot the error
"We sample , then set and backprop."
The
=1 is smuggled in — you can only claim it after writing . Sampling first and asserting the derivative later skips the entire trick."Reparameterize as so variance is correct."
Wrong scale. , not . You must multiply the standard deviation , not the variance.
"Encoder outputs , we plug it into ."
A standard deviation cannot be negative. Networks output an unconstrained value and pass it through (or softplus) so always holds.
" is drawn once at the start of training and reused."
Then becomes deterministic and the encoder learns a point estimate, collapsing the variational posterior. A fresh must be drawn every forward pass.
"For a -dim latent we draw one scalar and broadcast it."
That correlates all latent dimensions. Diagonal-covariance requires independent draws, , applied element-wise: .
"The trick eliminates the 4.3.01-KL-Divergence term from the ELBO."
No. Reparameterization only handles the reconstruction/expectation term's gradient. The KL term is separate and, for Gaussians, often has a closed form — untouched by the trick.
"Gradient of loss w.r.t. is computed directly."
It must go through the chain rule: . There is no direct dependence of on except through .
Why questions
Why can't gradients flow through a raw sampling operation?
Because sampling is a discontinuous, non-deterministic map: an infinitesimal change in produces no predictable change in the specific realized value, so no derivative exists at that node.
Why do we choose a standard normal as the base , rather than ?
The base must be parameter-free, so its density carries no . has fixed mean 0 and variance 1; all learnable shift/scale is loaded into the deterministic transform instead.
Why does the transform have to be differentiable and not just deterministic?
A deterministic-but-non-differentiable map (e.g. rounding) would still block the chain rule. We need to exist so backprop can multiply it in.
Why does the Gaussian's location–scale property matter specifically?
It guarantees is again Gaussian with the exact target mean and variance, so rewriting the sampler introduces no approximation — the two distributions are identical.
Why is single-sample training tolerable despite high variance?
Losses of smooth neural nets are locally smooth, and averaging over a mini-batch plus SGD's inherent noise-tolerance means the unbiased-but-noisy step still descends the objective on average.
Why prefer this estimator over the score-function estimator when both are available?
The pathwise estimator differentiates through , so it exploits the loss's slope directly — this usually slashes variance compared to weighting samples by -prob as in 6.1.02-Policy-Gradients.
Why is the Jacobian (diagonal case) just the identity?
Because depends only on and linearly, so , where (the Kronecker delta) is just shorthand for " when , otherwise" — i.e. each output responds only to its own mean. Diagonal, cheap, no cross-terms — that is what keeps VAE backprop fast.
Edge cases
What is when ?
exactly — a deterministic point. The distribution degenerates to a spike; still exists, but the model has stopped being stochastic in that dimension.
What happens to the gradient if the drawn ?
It is zero, since . That particular sample carries no information about ; other draws in the batch compensate.
If is very large (a rare tail draw), what breaks?
Nothing about correctness — the estimate stays unbiased — but lands far out, giving a high-magnitude, high-variance gradient. This is why gradient clipping and enough samples matter.
Can we reparameterize a Bernoulli/categorical latent this way?
Not with a smooth map — the output space is discrete, so no differentiable exists. You relax it continuously first, e.g. via the Gumbel-Softmax trick in 5.2.03-Gumbel-Softmax.
What if the covariance is full (not diagonal)?
Use where is the Cholesky factor of the covariance (), . Still deterministic and differentiable, just with a matrix multiply instead of element-wise scaling.
At the very start of training when are random, does the trick still work?
Yes. Correctness never depends on parameter quality — it only depends on the algebraic split. Poor initial just mean early gradients are noisy, not invalid.
Does reparameterization change the forward sampling result compared to naive sampling?
No. For matched it produces an identically distributed ; the change is purely about how derivatives are computed, not about the samples' statistics.
Recall One-line summary to lock in
The trick's job ::: move out of the random draw and into a differentiable transform, so the same expectation now has a chain-rule-friendly gradient. The single hardest trap ::: forgetting that reparameterization needs a continuous, differentiable base-to-target map — which is why discrete latents need Gumbel-Softmax instead.