Exercises — Generative Adversarial Networks (GAN) framework

(a) Both outputs are correct behaviour. $D$'s job: high on real, low on fake. $0.92$ on a real photo = "I think it's real" ✓. $0.15$ on a fake = "I think it's fake" ✓. A well-trained-against-a-weak-generator discriminator looks exactly like this. (b) At full convergence the generator matches the data, so $D$ genuinely cannot tell them apart. The safest guess for a coin it cannot read is $\boxed{0.5}$ — maximum uncertainty. It outputs $0.5$ on real and fake.

1. Discriminator — it should say "real" (high) on real data.
2. Generator — it wants its fakes to fool $D$ into saying "real" (high).
3. The term $\log (1-D(G(z)))$ is large when $D(G(z))$ is small (fake caught) and small when $D(G(z))$ is large (fake fools $D$). Since $G$ wants to fool $D$, $G$ wants this term small → $G$ minimises $V$. $D$ wants fakes caught → $D$ maximises $V$. Consistent with ${\min }_G{\max }_{D}V$.

Real term $\sum \log D(x)$: $\log 0.9+\log 0.8+\log 0.6+\log 0.7=-0.10536-0.22314-0.51083-0.35667=-1.19600$ Fake term $\sum \log (1-D(\tilde{x}))$, using $1-D=[0.8,0.9,0.7,0.6]$: $\log 0.8+\log 0.9+\log 0.7+\log 0.6=-0.22314-0.10536-0.35667-0.51083=-1.19600$ Sum $=-2.39200$. Then $L_D=-\frac{1}{4}(-2.39200)=\boxed{0.59800}.$

$\log 0.2+\log 0.1+\log 0.3+\log 0.4=-1.60944-2.30259-1.20397-0.91629=-6.03229$ $L_G=-\frac{1}{4}(-6.03229)=\boxed{1.50807}.$ Notice $L_G$ is large here: $D$ is confidently rejecting the fakes, so the generator is under heavy pressure — exactly the "strong gradient signal" the non-saturating trick was designed to keep.

(a) $D^{\ast }(x_0)=\frac{0.6}{0.6+0.2}=\frac{0.6}{0.8}=\boxed{0.75}.$ (b) At $x_0$ real data is 3× more likely than fake data, so the best any discriminator can do is be 75% confident it's real — that is the true posterior probability of "real". On the figure this is the orange dot at $r=1/3$. (c) If $p_{data}=p_g$ then $D^{\ast }=\frac{p}{p+p}=\frac{1}{2}=0.5$. The two distributions coincide → no information → forced coin-flip. This is precisely the convergence value from Exercise 1.1(b), and on the figure it is the green dot where the curve crosses $r=1$.

(a) — the algebra, and why it becomes a divergence. Start from the combined-integral form (parent note, change of variables $y=G(z)$): $V(G,D^{\ast })={\int }_x[p_{data}\log D^{\ast }+p_g\log (1-D^{\ast })]dx.$ Note $1-D^{\ast }=\frac{p_g}{p_{data}+p_g}$. Substitute both: $V={\int }_x[p_{data}\log \frac{p_{data}}{p_{data}+p_g}+p_g\log \frac{p_g}{p_{data}+p_g}]dx.$ The trick: we want a divergence, and divergences compare a distribution to the mixture $m=\frac{1}{2}(p_{data}+p_g)$. So force the denominator to be $2m$ by writing $p_{data}+p_g=2m$ and pulling out the $2$: $\log \frac{p_{data}}{p_{data}+p_g}=\log \frac{p_{data}}{2m}=\log \frac{p_{data}}{m}-\log 2.$ Do the same for the $p_g$ term. The two $-\log 2$ pieces collect (since $\int p_{data}=\int p_g=1$) into $-2\log 2=-\log 4$: $V=-\log 4+{\int }_x{p}_{data}\log \frac{p_{data}}{m}dx+{\int }_x{p}_g\log \frac{p_g}{m}dx.$ Those two remaining integrals are exactly $\mathrm{KL}(p_{data}\Vert m)$ and $\mathrm{KL}(p_g\Vert m)$, and their sum is $2\mathrm{JSD}$ by definition: $V(G,D^{\ast })=-\log 4+\mathrm{KL}(p_{data}\Vert m)+\mathrm{KL}(p_g\Vert m)=-\log 4+2\mathrm{JSD}(p_{data}\Vert p_g).$ Why a divergence appears: plugging in $D^{\ast }$ turned the score into "how far apart are $p_{data}$ and $p_g$, each measured against their average." That is a distance between distributions — the game literally minimises a statistical distance. (b) Since $\mathrm{JSD}\ge 0$, the smallest $V(G,D^{\ast })$ can be is when $\mathrm{JSD}=0$: $V_{\min }=-\log 4+2\cdot 0=-\log 4=-2\log 2\approx \boxed{-1.38629}.$ Achieved by the generator with $p_g=p_{data}$ — its output distribution exactly equals the real data distribution. That is the theoretical goal of the whole framework.

This is the vanishing-gradient failure. When $D(G(z))\approx 0$, the original generator objective $\log (1-D(G(z)))$ sits on a flat part of the log curve (its slope $\to 0$), so ${\nabla }_{{\theta }_g}$ is tiny — the generator receives almost no learning signal and freezes. Rescue: switch to the non-saturating loss, maximising $\log D(G(z))$ (minimise $L_G=-\log D(G(z))$). When $D(G(z))\approx 0$, the slope of $-\log D$ is huge ($\propto 1/D$), giving a strong gradient exactly when the generator most needs it. Same optimum ($D=0.5$), better dynamics. See the figure comparing the two curves.

At equilibrium $p_g=p_{data}$ so $D^{\ast }\to 0.5$ on both streams. A sensible rule with tolerance $ϵ$ and window $N$ epochs: $|{\bar{D}}_{real}-0.5|<ϵ\text{and}|{\bar{D}}_{fake}-0.5|<ϵ,\text{held for }N\text{ consecutive epochs.}$ Justification: the parent's numerical trace lands at ${\bar{D}}_{real}=0.58,{\bar{D}}_{fake}=0.54$ near convergence — both drifting toward $0.5$. Requiring both near $0.5$ prevents the false positive where a lazy $D$ outputs $0.5$ everywhere while $G$ is still bad. How to choose $ϵ$. Batch estimates of $\bar{D}$ are noisy: with batch size $m$ and per-sample variance $\le 0.25$ (a probability in $[0,1]$), the standard error of the mean is $\le \sqrt{0.25/m}=0.5/\sqrt{m}$. For $m=64$ this is $\approx 0.0625$. You want $ϵ$ a bit larger than the noise floor so genuine equilibrium is not rejected by random jitter, but smaller than the gap you saw during training ($0.58-0.54$ vs the early $0.91-0.12$). $ϵ\approx 0.03\text{–}0.05$ sits in that band for $m\gtrsim 100$. If your batch is smaller, raise $ϵ$ (more noise) or average $\bar{D}$ over several batches first (shrinks the noise floor by $1/\sqrt{\text{batches}}$). How to choose $N$. A single lucky epoch can pass by chance; requiring the condition for $N$ consecutive epochs makes a false trigger exponentially unlikely (if a random pass has probability $p$ per epoch, $N$ in a row has probability $\approx p^N$). $N\approx 5\text{–}10$ is typical — long enough to reject flukes, short enough not to overshoot into mode collapse. Sensitivity: the rule is robust — too-small $ϵ$ or too-large $N$ merely makes you stop later (safe); too-large $ϵ$ or too-small $N$ risks stopping early on a transient, so err on the strict side.

Differentiate w.r.t. $d$: $f^{\prime }(d)=\frac{a}{d}-\frac{b}{1-d}.$ Set $f^{\prime }(d)=0$: $\frac{a}{d}=\frac{b}{1-d}⟹a(1-d)=bd⟹a=d(a+b)⟹d^{\ast }=\frac{a}{a+b}.$ Second derivative: $f^{\prime \prime }(d)=-\frac{a}{d^2}-\frac{b}{(1-d{)}^2}<0(\text{since }a,b>0).$ Why concavity forces a maximum (the intuition). $f^{\prime }(d)=0$ only tells us the curve is momentarily flat at $d^{\ast }$ — that flat point could be a peak, a valley, or a shoulder. The second derivative reads the bending: $f^{\prime \prime }<0$ means the slope is always decreasing as $d$ grows. So just left of $d^{\ast }$ the slope is positive (curve rising), and just right of $d^{\ast }$ the slope is negative (curve falling). A curve that rises then falls has a peak at the turning point — never a valley. Because $f^{\prime \prime }<0$ for every $d$ in $(0,1)$ (not just at $d^{\ast }$), there is no second turning point and no minimum hiding elsewhere: $d^{\ast }$ is the unique global maximum. Substituting $a=p_{data},b=p_g$ recovers $D^{\ast }(x)=\frac{p_{data}(x)}{p_{data}(x)+p_g(x)}$. $■$

Optimal $D^{\ast }$ at each point ($D^{\ast }=\frac{p_{data}}{p_{data}+p_g}$):

• $A$: $D^{\ast }=0.7/(0.7+0.3)=0.7$; $B$: $D^{\ast }=0.3/(0.3+0.7)=0.3$.

Direct $V$. The expectation over a discrete world is a plain weighted sum. The first term ${\mathbb{E}}_{p_{data}}[\log D^{\ast }]$ weights by $p_{data}$; the second ${\mathbb{E}}_{p_g}[\log (1-D^{\ast })]$ weights by $p_g$: $V=[p_{data}(A)\log D_A^{\ast }+p_{data}(B)\log D_B^{\ast }]+[p_g(A)\log (1-D_A^{\ast })+p_g(B)\log (1-D_B^{\ast })].$ With $1-D_A^{\ast }=0.3,1-D_B^{\ast }=0.7$: $V=[0.7\log 0.7+0.3\log 0.3]+[0.3\log 0.3+0.7\log 0.7]=2(0.7\log 0.7+0.3\log 0.3).$ $=2(-0.24941-0.36119)=2(-0.61060)=-\mathrm{1.22120.}$ Via JSD. $M=\frac{1}{2}(P+Q)$: at $A$, $M=\frac{1}{2}(0.7+0.3)=0.5$; at $B$, $M=0.5$. $\mathrm{KL}(P\Vert M)=0.7\log \frac{0.7}{0.5}+0.3\log \frac{0.3}{0.5}=0.7(0.33647)+0.3(-0.51083)=0.23553-0.15325=\mathrm{0.08228.}$ By symmetry $\mathrm{KL}(Q\Vert M)=0.08228$, so $\mathrm{JSD}=\frac{1}{2}(0.08228+0.08228)=\mathrm{0.08228.}$ $-\log 4+2\mathrm{JSD}=-1.38629+2(0.08228)=-1.38629+0.16457=-\mathrm{1.22172.}$ Direct $-1.22120$ vs JSD-form $-1.22172$ — they agree (the tiny gap is rounding; in exact arithmetic they are identical). ✓ The value function at the optimal $D$ really is a shifted Jensen–Shannon divergence.

Which convergence value do all these exercises point to? ::: $D=0.5$ on both real and fake, where $p_g=p_{data}$ and $V(G,D^{\ast })=-\log 4$. Which loss form fixes vanishing generator gradients? ::: Non-saturating: maximise $\log D(G(z))$.