Exercises — Generator and discriminator dynamics

What we do: look at each term and ask "does it contain $G$?".

• Term one, ${\mathbb{E}}_{x\sim p_{data}}[\log D(x)]$, only involves real samples $x$ and the network $D$. There is no $G$ anywhere.
• Term two, ${\mathbb{E}}_{z\sim p_z}[\log (1-D(G(z)))]$, contains $G(z)$ explicitly.

Why the first term vanishes: when we take the gradient with respect to $G$'s parameters ${\theta }_g$, anything not containing ${\theta }_g$ is a constant, and the derivative of a constant is $0$. ${\nabla }_{{\theta }_g}V={\nabla }_{{\theta }_g}{\mathbb{E}}_z[\log (1-D(G(z)))].$ Answer: $G$ only sees the second term, ${\mathbb{E}}_z[\log (1-D(G(z)))]$.

$\log (0.9)=-0.10536$. $D$ maximizes $V$, so it wants this term as large (closest to $0$) as possible. Pushing $D(x)\to 1$ pushes $\log D(x)\to 0$ (the largest it can be, since $\log$ of a probability is $\le 0$). So larger is better for $D$ — and larger means more confident the real image is real. ✓

Original (minimax) term: $\log (1-0.05)=\log (0.95)=-\mathrm{0.05129.}$ This is already very close to $0$ — meaning $G$'s objective is nearly flat here, so its slope (gradient) is tiny. Bad news early on.

Non-saturating loss: $-\log (0.05)=\mathrm{2.9957.}$ A large positive loss → a large gradient → $G$ actually learns. That's the whole point of switching objectives. See 4.5.06-GAN-training-algorithm for where these plug into the loop.

$D$'s objective value on this pair: $\log (0.95)+\log (1-0.15)=\log (0.95)+\log (0.85)=-0.05129+(-0.16252)=-\mathrm{0.21381.}$ $D$'s loss is the negative of that: $-(-0.21381)=\mathrm{0.21381.}$ A small loss → $D$ is doing well. This is exactly negative binary cross-entropy with real label $1$ and fake label $0$.

The standard normal density is $\varphi (u)=\frac{1}{\sqrt{2\pi }}{e}^{-u^2/2}$.

• $p_{data}(1)=\varphi (1-0)=\frac{1}{\sqrt{2\pi }}{e}^{-1/2}=0.24197$.
• $p_g(1)=\varphi (1-2)=\frac{1}{\sqrt{2\pi }}{e}^{-1/2}=0.24197$ (same, since $x=1$ is exactly halfway).

$D^{\ast }(1)=\frac{0.24197}{0.24197+0.24197}=\frac{1}{2}=\mathrm{0.5.}$ Interpretation: at the crossover point where real and fake are equally likely, even the best possible discriminator is reduced to a coin flip. Off to the sides it recovers confidence. See figure below.

Step 1 — $D^{\ast }$ collapses to a half. Put $p_g=p_{data}$ into the formula: $D^{\ast }(x)=\frac{p_{data}(x)}{p_{data}(x)+p_{data}(x)}=\frac{p_{data}(x)}{2p_{data}(x)}=\frac{1}{2},$ valid wherever $p_{data}(x)>0$ (i.e. everywhere data lives).

Step 2 — plug into $V$. With $D=\frac{1}{2}$: $V={\mathbb{E}}_{p_{data}}[\log \frac{1}{2}]+{\mathbb{E}}_{p_g}[\log (1-\frac{1}{2})]=\log \frac{1}{2}+\log \frac{1}{2}=2\log \frac{1}{2}.$ $2\log (0.5)=-1.38629=-\log 4.$ Meaning: the game "settles" at value $-\log 4$. This is the Nash equilibrium — neither player can improve by moving alone.

What we do & why: for a fixed $x$, choosing $D(x)$ is just choosing one number to maximize $f$. Calculus maximization: set the derivative to $0$. $\frac{df}{dD}=\frac{p_{data}}{D}-\frac{p_g}{1-D}=0.$ The two terms come from $\frac{d}{dD}\log D=\frac{1}{D}$ and $\frac{d}{dD}\log (1-D)=\frac{-1}{1-D}$.

Solve: $\frac{p_{data}}{D}=\frac{p_g}{1-D}\Rightarrow p_{data}(1-D)=p_{g}D\Rightarrow p_{data}=D(p_{data}+p_g).$ $\boxed{D^{\ast }(x)=\frac{p_{data}(x)}{p_{data}(x)+p_g(x)}.}$

Is it a max not a min? The second derivative $\frac{d^{2}f}{d{D}^2}=-\frac{p_{data}}{D^2}-\frac{p_g}{(1-D{)}^2}<0$, so $f$ is concave — the stationary point is a maximum. ✓

Setup. Let $m(x)=\frac{1}{2}(p_{data}(x)+p_g(x))$, the mixture of real and fake. Then $D^{\ast }=\frac{p_{data}}{2m}$ and $1-D^{\ast }=\frac{p_g}{2m}$.

Substitute: $V(D^{\ast },G)=\int p_{data}\log \frac{p_{data}}{2m}dx+\int p_g\log \frac{p_g}{2m}dx.$ Split each $\log \frac{\cdot }{2m}=\log \frac{\cdot }{m}-\log 2$: $=\underset{KL(p_{data}\Vert m)}{\underbrace{\int p_{data}\log \frac{p_{data}}{m}}}+\underset{KL(p_g\Vert m)}{\underbrace{\int p_g\log \frac{p_g}{m}}}-\log 2\int p_{data}-\log 2\int p_g.$ Both densities integrate to $1$, so the last two terms give $-2\log 2=-\log 4$.

Recognize JSD. By definition $JSD(p_{data}\Vert p_g)=\frac{1}{2}KL(p_{data}\Vert m)+\frac{1}{2}KL(p_g\Vert m)$, so the two KL terms sum to $2JSD$: $V(D^{\ast },G)=2JSD(p_{data}\Vert p_g)-\log 4.$ Minimum: $JSD\ge 0$ with equality iff $p_{data}=p_g$. So the smallest $V$ is $-\log 4$, reached exactly at $p_g=p_{data}$ — matching Exercise 3.2. ∎

The JSD failure. From 4.2, the optimal-$D$ generator objective is (up to constants) $JSD(p_{data}\Vert p_g)$. Suppose the fake and real distributions have disjoint supports — e.g. images on two thin manifolds that don't touch (extremely common early in training, and generic for high-dimensional data). Then for the mixture $m$:

• wherever $p_{data}>0$ we have $p_g=0$, so $\log \frac{p_{data}}{m}=\log \frac{p_{data}}{p_{data}/2}=\log 2$;
• symmetrically for $p_g$.

So $JSD=\log 2$, a constant, regardless of how far apart the two supports are. A constant has zero gradient — moving $p_g$ closer to $p_{data}$ produces no signal until they actually overlap. That is precisely the vanishing-gradient pathology.

WGAN's cure. The Wasserstein (earth-mover) distance measures the minimum cost of transporting mass from $p_g$ to $p_{data}$. Unlike JSD, it is continuous and gives a distance proportional to the actual separation: two disjoint blobs a distance $d$ apart have $W=d$, not a constant. Its gradient therefore always points toward $p_{data}$, even with no overlap. That's why WGAN training is smoother and less prone to the dead-gradient regime.

Why the objective permits collapse. $G$ minimizes ${\mathbb{E}}_z[\log (1-D(G(z)))]$ — an expectation over individual samples. Nothing in this term rewards diversity across the batch. If a single point $x^{\star }$ happens to satisfy $D(x^{\star })\approx 1$, then producing only $x^{\star }$ drives the objective as low as producing a rich variety. The loss is blind to how many distinct modes $G$ covers; it only cares about per-sample foolingness.

A fix — minibatch discrimination / feature-matching. Give $D$ (or the loss) access to batch-level statistics so it can detect "all these samples are suspiciously identical". Concretely, feature-matching sets ${\mathcal{L}}_G=\Vert {\mathbb{E}}_{x\sim p_{data}}[f(x)]-{\mathbb{E}}_z[f(G(z))]{\Vert }^2,$ where $f$ is an intermediate $D$-feature. This forces the distribution of $G$'s features to match the distribution of real features — a single repeated output can't reproduce the spread/variance of real features, so it is penalized.

Why this targets the cause: the root problem was that the objective operated per-sample and ignored the set of outputs. Both minibatch discrimination and feature-matching lift the signal from single samples to batch statistics, restoring a term that explicitly cares about diversity. WGAN also helps indirectly by giving healthier gradients, but does not itself forbid collapse — the batch-statistics term does.

L1 which term does G depend on ::: only ${\mathbb{E}}_z[\log (1-D(G(z)))]$ L2 non-saturating loss at $D(G(z))=0.05$ ::: $-\log 0.05=2.996$ L3 optimal $D$ at the midpoint of two equal-variance Gaussians ::: $0.5$ L3 game value at equilibrium ::: $-\log 4\approx -1.386$ L4 generator objective under $D^{\ast }$ ::: $2JSD(p_{data}\Vert p_g)-\log 4$ L5 why JSD gives no gradient on disjoint supports ::: it locks to the constant $\log 2$