Generative Adversarial Networks (GAN) framework: Worked examples

This page is a drill deck. The parent note built the value function $V (G, D) = E_{x \sim p_{d a t a}} [lo g D (x)] + E_{z \sim p_{z}} [lo g (1 - D (G (z)))]$ and the optimal discriminator. Here we use those formulas on every case that training can throw at you.

Before we start, one symbol reminder so no notation is used blind:

The scenario matrix

Cell	Case class	What breaks / what to watch	Example
A	$p_{d a t a} = p_{g}$ (perfect convergence)	$D^{*} = 0.5$ everywhere, $V = - lo g 4$	Ex 1
B	$p_{g}$ far off (early training)	$D^{*} \to 1$ on real, $\to 0$ on fake	Ex 2
C	Sign / direction of the gradient	minimax vs non-saturating loss	Ex 3
D	Degenerate: $p_{d a t a} (x) = 0$ (fake-only region)	$D^{*} = 0$ , division safe?	Ex 4
E	Degenerate: $p_{g} (x) = 0$ (mode dropped)	$D^{*} = 1$ , links to Mode Collapse Solutions	Ex 5
F	Limiting: disjoint supports	JSD saturates → vanishing gradient, Waserstein GAN (WGAN)	Ex 6
G	Real-world word problem (image batch)	plug numbers into losses	Ex 7
H	Exam twist: verify $V = - lo g 4$ from JSD	algebra of the theorem	Ex 8

We now hit every cell.

Worked examples

Forecast: guess before reading — should the perfect detective say "real", "fake", or shrug?

Plug into the master formula. $D^{*} (x_{0}) = \frac{p _{d a t a} ( x _{0} )}{p _{d a t a} ( x _{0} ) + p _{g} ( x _{0} )} = \frac{0.3}{0.3 + 0.3}$ Why this step? $D^{*}$ is a ratio "how much of the stuff at $x_{0}$ is real, out of all the stuff at $x_{0}$ ". Half is real, half is fake.
Simplify. $D^{*} (x_{0}) = \frac{0.3}{0.6} = 0.5$ . Why this step? Equal numerator-and-denominator-half always gives $0.5$ — the shrug.

Verify: the detective outputting $0.5$ is exactly the equilibrium the parent note promised: maximum uncertainty, the counterfeiter has won. ✓ ( $D^{*} = 0.5$ )

Forecast: which one gets pushed toward $1$ , which toward $0$ ?

Real point. $D^{*} (x_{r}) = \frac{0.40}{0.40 + 0.01} = \frac{0.40}{0.41} \approx 0.9756$ Why this step? At $x_{r}$ almost everything present is real, so the ratio is near $1$ .
Fake point. $D^{*} (x_{f}) = \frac{0.02}{0.02 + 0.50} = \frac{0.02}{0.52} \approx 0.0385$ Why this step? At $x_{f}$ almost everything is fake, so the "fraction real" is tiny.

Verify: $D^{*} (x_{r}) \approx 0.976$ (near 1) and $D^{*} (x_{f}) \approx 0.038$ (near 0) — this matches the parent's "Epoch 10" trace ( $D (x_{r e a l}) \approx 0.91$ , $D (G (z)) \approx 0.12$ ): an easy job for $D$ when $G$ is weak. ✓

Forecast: which loss gives the poor generator a stronger push?

Look at the shape of both curves before computing.

Figure — Generative Adversarial Networks (GAN) framework

Original loss slope. The generator minimizes $lo g (1 - d)$ , so its gradient (w.r.t. $d$ ) is $\frac{d}{dd} lo g (1 - d) = \frac{- 1}{1 - d}_{d = 0.05} = \frac{- 1}{0.95} \approx - 1.0526.$ Why this step? The slope size tells us how hard $d$ is pushed. Look at the pink curve on the figure: near $d = 0$ it is almost flat — small slope, weak signal.
Non-saturating loss slope. The generator maximizes $lo g d$ , gradient $\frac{d}{dd} lo g d = \frac{1}{d}_{d = 0.05} = \frac{1}{0.05} = 20.$ Why this step? The blue curve is steep near $d = 0$ : magnitude $20$ vs $1.05$ — about 19× stronger.
Same fixed point. Both are optimized when $d \to$ larger (generator wants $D$ fooled); crucially both agree $d = 0.5$ is where $D$ is confused, but the blue one fights harder when $d$ is small. Why this step? This is the parent note's "vanishing gradient" fix in one number.

Verify: ratio $\frac{20}{1.0526} = 19.0$ . The non-saturating loss is ~19× stronger here. ✓

Forecast: a spot with no real support — what should the detective say?

Plug in. $D^{*} (x_{1}) = \frac{0}{0 + 0.25} = \frac{0}{0.25} = 0.$ Why this step? Numerator is $0$ but the denominator $0.25 \neq = 0$ , so no division by zero — the ratio is cleanly $0$ .
Interpret. $D^{*} = 0$ = "certainly fake", correct: nothing real lives here. Why this step? This is the only place $D$ can be perfectly confident and be right, so $G$ gets a huge push to abandon $x_{1}$ .

Verify: $D^{*} (x_{1}) = 0$ , and the denominator was nonzero so the formula is well-defined. ✓

Forecast: a real mode the generator ignores — confident or confused?

Plug in. $D^{*} (x_{2}) = \frac{0.18}{0.18 + 0} = \frac{0.18}{0.18} = 1.$ Why this step? All mass here is real, so fraction-real $= 1$ : "surely real".
Interpret. $D^{*} = 1$ means $D$ flags this region as always real — the generator gets zero reward for producing it, so it has no direct pull back to cover "7". This is precisely the failure Mode Collapse Solutions attacks. Why this step? $D^{*} = 1$ is the fingerprint of a dropped mode in the matrix.

Verify: $D^{*} (x_{2}) = 1$ (denominator $0.18 \neq = 0$ , safe). ✓

Forecast: if the two curves never touch, can the generator ever get a useful gradient?

On $R$ (real only): $p_{g} = 0$ so $D^{*} = \frac{p _{d a t a}}{p _{d a t a} + 0} = 1$ . On $F$ (fake only):** $p_{d a t a} = 0$ so $D^{*} = \frac{0}{0 + p _{g}} = 0$ . Why this step? $D$ becomes a perfect classifier — it is never wrong, so it never leaks information about direction to $G$ .
JSD is maxed. For fully disjoint distributions $JSD = lo g 2$ , so $V (G, D^{*}) = - lo g 4 + 2 lo g 2 = - lo g 4 + lo g 4 = 0.$ Why this step? $JSD = lo g 2 \approx 0.6931$ is its maximum; the value function pins to $0$ and its gradient w.r.t. $G$ vanishes — this is the theoretical reason Waserstein GAN (WGAN) replaces JSD with a distance that still slopes across a gap.

Verify: $- lo g 4 + 2 lo g 2 = 0$ (since $lo g 4 = 2 lo g 2$ ). ✓ And $JSD = lo g 2 \approx 0.6931$ . ✓

Forecast: is the discriminator doing well (small $L_{D}$ ) at this snapshot?

Real-term sum. $lo g 0.9 + lo g 0.8 + lo g 0.6 + lo g 0.7$ $\approx (- 0.10536) + (- 0.22314) + (- 0.51083) + (- 0.35667) = - 1.19600$ . Why this step? Each $lo g D (x)$ rewards confidence-on-real; closer to $0$ is better.
Fake-term sum. $lo g (1 - 0.2) + lo g (1 - 0.3) + lo g (1 - 0.1) + lo g (1 - 0.4)$ $= lo g 0.8 + lo g 0.7 + lo g 0.9 + lo g 0.6$ $\approx (- 0.22314) + (- 0.35667) + (- 0.10536) + (- 0.51083) = - 1.19600$ . Why this step? Rewards confidence-on-fake ( $1 - D$ near $1$ ).
Combine. $L_{D} = - \frac{1}{4} (- 1.19600 - 1.19600) = - \frac{1}{4} (- 2.39200) = 0.59800$ . Why this step? The $- \frac{1}{m}$ turns "reward" into "loss to minimize".
Generator loss. $lo g 0.2 + lo g 0.3 + lo g 0.1 + lo g 0.4$ $\approx (- 1.60944) + (- 1.20397) + (- 2.30259) + (- 0.91629) = - 6.03229$ . $L_{G} = - \frac{1}{4} (- 6.03229) = 1.50807$ . Why this step? $L_{G}$ is large because $D$ still catches the fakes ( $D (\tilde{x})$ small) — the generator has work to do.

Verify: $L_{D} \approx 0.5980$ , $L_{G} \approx 1.5081$ . Both positive (losses), and $L_{G} > L_{D}$ signals a strong discriminator winning the round. ✓ (See Cross-Entropy Loss — this is averaged binary cross-entropy.)

Forecast: what happens to JSD when two identical curves are compared?

JSD of a curve with itself is $0$ . By definition $JSD (p ∥ p) = 0$ (a distance from something to itself). Why this step? JSD is a divergence: zero iff the distributions match, and here they match exactly.
Substitute. $V = - lo g 4 + 2 \cdot 0 = - lo g 4$ . Why this step? The bonus term vanishes, leaving the constant floor.
Cross-check via $D^{*}$ . With $p_{g} = p_{d a t a}$ , $D^{*} = \frac{p}{p + p} = \frac{1}{2}$ . Then $V = E [lo g \frac{1}{2}] + E [lo g (1 - \frac{1}{2})] = lo g \frac{1}{2} + lo g \frac{1}{2} = 2 lo g \frac{1}{2} = - 2 lo g 2 = - lo g 4.$ Why this step? Two independent routes give the same number — this is the Nash Equilibrium value of the game.

Verify: $- lo g 4 \approx - 1.38629$ and $2 lo g (1/2) \approx - 1.38629$ — identical. ✓

Recall Self-test: which cell?

$D^{*} = 1$ at a point means... ::: the generator has dropped that mode ( $p_{g} = 0$ there) — Cell E. Disjoint supports give JSD = ... and $V (G, D^{*}) =$ ... ::: $lo g 2$ , and $0$ ; gradients vanish (Cell F). The non-saturating generator loss exists to fix... ::: weak/vanishing gradients when $D (G (z))$ is small (Cell C). At convergence $D^{*} =$ ... and $V =$ ... ::: $0.5$ and $- lo g 4$ (Cells A/H).