Worked examples — Generator and discriminator dynamics

This deep dive is the practice arena for the parent note on GAN dynamics. There we built the machinery: the objective $V(D,G)$, the optimal discriminator $D^{\ast }$, the vanishing-gradient trap, and the equilibrium. Here we throw every kind of scenario at that machinery and solve each one completely.

Before we start, one promise: every symbol below was earned in the parent, but we re-anchor the critical ones so you never feel lost.

Recall The six symbols we lean on

• $D(x)$ ::: the discriminator's output — a number between $0$ and $1$, "probability this input is REAL".
• $G(z)$ ::: the generator turning a random noise vector $z$ into a fake sample.
• $p_{data}(x)$ ::: how likely real data is to look like $x$ (a probability density).
• $p_g(x)$ ::: how likely the generator is to produce something looking like $x$.
• $D^{\ast }$ (read "D-star") ::: the best possible discriminator for a fixed generator; from the parent $D^{\ast }(x)=\frac{p_{data}(x)}{p_{data}(x)+p_g(x)}$.
• $V(D,G)$ ::: the game's score for the discriminator — bigger is better for D, smaller is better for G.

---

The scenario matrix

Every problem this topic can throw sits in one of these cells. Think of it like a chessboard: we must visit each square at least once. The table has two axes: the rows are the training regime / probability structure (what state the game is in), and the columns are what makes the cell special (which extreme, zero, or twist it isolates).

Cell	Row: regime	Column: what is extreme / degenerate / applied	Which example
A	Early training	$D(G(z))\approx 0$ — G failing, gradient vanishes	Ex 1
B	Late training	$D(G(z))\approx 1$ — G winning	Ex 2
C	Perfect equilibrium	$p_g=p_{data}$, so $D^{\ast }=\frac{1}{2}$ everywhere	Ex 3
D	Optimal $D^{\ast }$ at a point	plug densities into $D^{\ast }(x)$ formula	Ex 4
E	Degenerate zero	$p_{data}(x)=0$ or $p_g(x)=0$ at a point	Ex 5
F	Gradient regime	saturating vs non-saturating gradient, compared numerically	Ex 6
G	Real-world word problem	fraud/counterfeit framing, compute $V$	Ex 7
H	Exam twist	value of the game at $D^{\ast }$ equals $-\log 4$	Ex 8
I	Mode collapse limiting case	G ignores a mode, measure the miss	Ex 9

Read the table down: rows A–B fix who is winning, C–E fix the probability structure (matched, single point, degenerate zero), F fixes the gradient regime, and G–I are applied and exam-style variants. Visiting every row → covering the topic.

Prerequisite links you may want open: binary cross-entropy, Nash equilibrium, and for where this leads, mode collapse and Wasserstein GAN.

The figure above is our map: each region is a cell of the matrix, plotted on the "value of $D(G(z))$" line from $0$ (D sure it's fake) to $1$ (D fooled).

---

Cell A — Early training, G is failing

Forecast: guess before reading — will the original loss be near $0$ or a big number? (It will be near $0$ — that's the whole problem.)

1. Original loss value. $\log (1-0.05)=\log (0.95)$. Why this step? The parent's minimax objective makes G minimize $\log (1-D(G(z)))$; we evaluate it at the current D-output to see how "alive" it is. $\log (0.95)\approx -0.0513$ This is tiny — nearly flat. A flat loss means a tiny slope, means almost no learning signal.

2. Non-saturating loss value. $-\log D(G(z))=-\log (0.05)$. Why this step? The parent's fix was to maximize $\log D(G(z))$, i.e. minimize $-\log D(G(z))$. We evaluate the alternative at the same point to compare. $-\log (0.05)\approx 2.9957$ A big number — steep, lots of signal.

3. Interpret. The original sits at $-0.051$ (flat lake), the non-saturating at $+3.0$ (steep cliff). G learns from the cliff, not the lake. Why this step? Learning speed is proportional to slope; the non-saturating loss keeps a strong slope exactly when $D(G(z))\to 0$.

Verify: sanity check with logs — $\log (0.95)\approx -0.051$ and $-\log (0.05)=\log (20)\approx 3.0$. Ratio of magnitudes $\approx 58\times$ stronger for non-saturating. Units: both are dimensionless log-probabilities. ✓

---

Cell B — Late training, G is winning

Forecast: G is doing well now — should its original loss be very negative or near zero?

1. G's original loss. $\log (1-0.9)=\log (0.1)$. Why this step? This is the term G minimizes; a very negative value means G is happy (it drove D's fake-probability up). $\log (0.1)\approx -2.3026$ Strongly negative → G is succeeding.

2. D's contribution for this fake. For a fake, D wants $\log (1-D(G(z)))$ large (close to $0$), but here it is $-2.30$, deeply negative. Why this step? D maximizes $V$; a big negative term means D is being punished — it got fooled.

3. Contrast with Ex 1. In cell A the same term was $-0.05$ (D happy). Here $-2.30$ (D unhappy). The tables turned. Why this step? Placing the two extremes side by side shows the term's full range — near-zero when G fails, deeply negative when G wins — so no value in between will surprise the reader.

Verify: $\log (0.1)=-\ln 10\approx -2.3026$. Consistent with $e^{-2.3026}\approx 0.1$. ✓

---

Cell C — Perfect equilibrium

Forecast: if D outputs exactly $0.5$, what is $\log (1-0.5)$? Same as $\log (0.5)$ — so both terms coincide.

1. Plug into $D^{\ast }$. From the parent, $D^{\ast }(x)=\frac{p_{data}(x)}{p_{data}(x)+p_g(x)}$. Set $p_g=p_{data}$: $D^{\ast }(x)=\frac{p_{data}(x)}{p_{data}(x)+p_{data}(x)}=\frac{p_{data}(x)}{2p_{data}(x)}=\frac{1}{2}.$ Why this step? Equilibrium in a Nash sense means neither player can improve; the parent showed this happens precisely when the two densities match, forcing the ratio to $\frac{1}{2}$.

2. This holds for all $x$ where $p_{data}(x)>0$ — the $p_{data}(x)$ cancels regardless of its value. Why this step? We must cover every $x$, not one point; the cancellation is universal.

3. Losses at $D=0.5$. Real term $\log (0.5)\approx -0.6931$; fake term $\log (1-0.5)=\log (0.5)\approx -0.6931$. Why this step? At equilibrium D can do no better than guessing — its per-sample log-score bottoms out at $\log \frac{1}{2}$.

4. Averaged value $V$. Using this page's definition $V=\frac{1}{2}(\log D(x)+\log (1-D(G(z))))$: $V(D^{\ast },G)=\frac{1}{2}(\log 0.5+\log 0.5)=\frac{1}{2}(-0.6931-0.6931)=-\mathrm{0.6931.}$ Why this step? We stated $V$ in per-sample averaged form up top; showing its equilibrium number here makes that floor concrete and gives Ex 7 a benchmark to compare against.

Verify: $\frac{p}{2p}=0.5$ for any $p\ne 0$, $\log (0.5)=-\ln 2\approx -0.6931$, and the averaged $V=\log (0.5)\approx -0.6931$. ✓

---

Cell D — Optimal $D^{\ast }$ at a specific point

Forecast: $x=1$ is exactly halfway between the two means (0 and 2). Guess: $D^{\ast }\approx 0.5$.

1. Densities at $x=1$. Using the Gaussian density $\mathcal{N}(\mu ,1)$ defined at the top of this page, $p(x)=\frac{1}{\sqrt{2\pi }}{e}^{-(x-\mu {)}^2/2}$. $p_{data}(1)=\frac{1}{\sqrt{2\pi }}{e}^{-(1-0{)}^2/2}=\frac{1}{\sqrt{2\pi }}{e}^{-0.5}\approx 0.2420$ $p_g(1)=\frac{1}{\sqrt{2\pi }}{e}^{-(1-2{)}^2/2}=\frac{1}{\sqrt{2\pi }}{e}^{-0.5}\approx 0.2420$ Why this step? $D^{\ast }$ needs both densities; by symmetry the exponents match at the midpoint.

2. Plug into $D^{\ast }$. $D^{\ast }(1)=\frac{0.2420}{0.2420+0.2420}=\frac{1}{2}.$ Why this step? Equal densities → the Bayes-optimal detector is a coin flip; it genuinely cannot decide.

3. Compare to parent's endpoints ($D^{\ast }(0)\approx 0.88$, $D^{\ast }(2)\approx 0.12$). At the midpoint, $0.5$ — a smooth crossover. Why this step? Checking the midpoint against the two centres confirms $D^{\ast }$ varies monotonically across the axis (real-confident at $0$, coin-flip at $1$, fake-confident at $2$), so we have mapped the detector's behaviour over the whole line, not just one point.

Verify: exponents $-(1{)}^2/2$ and $-(-1{)}^2/2$ are both $-0.5$, so the fraction is exactly $\frac{1}{2}$. ✓

---

Cell E — Degenerate zero density

Forecast: if there is zero chance the sample is fake, D should be 100% sure it's real.

1. Plug in the zero. $D^{\ast }(x_0)=\frac{p_{data}(x_0)}{p_{data}(x_0)+p_g(x_0)}=\frac{0.3}{0.3+0}=1.$ Why this step? The formula still applies as long as the denominator is nonzero; here it is $0.3$.

2. The other degenerate direction. If instead $p_{data}(x_1)=0$ but $p_g(x_1)=0.3$ (generator hallucinates something real data never does): $D^{\ast }(x_1)=\frac{0}{0+0.3}=0.$ Why this step? We must cover both signs of degeneracy — a $0$ in the numerator vs a $0$ in the second density.

3. The truly undefined case. If both $p_{data}(x_2)=0$ and $p_g(x_2)=0$, the fraction is $\frac{0}{0}$ — undefined. D never sees such points, so its behaviour there is arbitrary; this is exactly the "empty region" where GANs behave unpredictably. Why this step? Limiting/degenerate inputs are a matrix cell we must not skip.

4. Warning sign. $D^{\ast }(x_0)=1$ means D is perfectly confident — a region where G contributes nothing. If large regions have $p_g=0$, G is missing whole chunks of the data: the seed of mode collapse.

Verify: $\frac{0.3}{0.3}=1$ and $\frac{0}{0.3}=0$. The $\frac{0}{0}$ case is correctly flagged undefined. ✓

---

Cell F — Gradient comparison, the two G-losses

Forecast: at low $D(G(z))$ the non-saturating should dominate; near $0.95$, the original should.

1. Define the ratio $R=\frac{\text{non-sat}}{\text{orig}}=\frac{1/D}{1/(1-D)}=\frac{1-D}{D}$, writing $D$ for $D(G(z))$. Why this step? A ratio strips away the shared $D^{\prime }{G}^{\prime }$ factor, isolating which regime each loss favours.

2. At $D=0.05$: $R=\frac{0.95}{0.05}=19$. Non-saturating is $19\times$ stronger. At $D=0.5$: $R=\frac{0.5}{0.5}=1$. They tie. At $D=0.95$: $R=\frac{0.05}{0.95}\approx 0.0526$. Now the original is $\sim 19\times$ stronger. Why this step? We test all three sub-regimes (failing, tied, winning) — that is what "cover every case" demands.

3. Interpret. Early on ($D$ small, G failing) the non-saturating loss wins — precisely when we need signal. This is why the non-saturating loss $-\log D(G(z))$ is the one practitioners actually train with: it keeps a strong gradient in exactly the regime where the theoretically-correct original loss goes flat and stalls learning. Why this step? The ratio $R$ is a raw number; connecting it to the practical training choice explains why we computed it and stands on its own without needing the parent open.

The figure above plots the two gradient-magnitude factors, $\frac{1}{1-D}$ (original, orange) and $\frac{1}{D}$ (non-saturating, magenta), across all $D(G(z))$ from $0$ to $1$. Read it left-to-right: on the left (G failing, $D$ small) the magenta curve rockets up while orange stays flat near $1$ — the non-saturating loss delivers a huge push exactly when the original goes flat. The two dashed violet lines mark our three test points; they cross at $D=0.5$, the tie from step 2. On the right (G winning, $D$ near $1$) the roles swap: orange climbs, magenta flattens. The picture is step 2, made continuous.

Verify: $R=\frac{1-D}{D}$ gives $19$, $1$, and $\frac{1}{19}\approx 0.0526$ at $D=0.05,0.5,0.95$. ✓

---

Cell G — Real-world word problem

Forecast: D is right-ish on both (high for real, low-ish for fake) — expect a modestly negative $V$ (log scores are always $\le 0$).

1. Real term. $\log D(x)=\log (0.8)\approx -0.2231$. Why this step? This is the first objective term — reward for calling real "real".

2. Fake term. $\log (1-D(G(z)))=\log (1-0.3)=\log (0.7)\approx -0.3567$. Why this step? Second objective term — reward for calling the counterfeit "fake"; higher $1-D$ is better for D.

3. Average with the $\frac{1}{2}$. $V=\frac{1}{2}(-0.2231+-0.3567)=\frac{1}{2}(-0.5798)\approx -0.2899$. Why this step? We use the page's $V$ definition — the $\frac{1}{2}$-averaged form — so this single number is directly comparable to the equilibrium value computed in Ex 3.

4. Winning? From Ex 3 the equilibrium floor is $V^{\text{eq}}=-0.6931$. Since $-0.2899>-0.6931$, D scores above the coin-flip floor — the detective is winning this batch: it is doing measurably better than blind guessing. Why this step? "Winning" only means something relative to the equilibrium baseline; comparing our $V$ to that floor turns a bare number into a verdict.

Verify: $\log (0.8)\approx -0.2231$, $\log (0.7)\approx -0.3567$, mean $\approx -0.2899$, which exceeds $-0.6931$. ✓

---

Cell H — Exam twist: the value of the game

Forecast: two terms, each $\log \frac{1}{2}$; summed → $2\log \frac{1}{2}=\log \frac{1}{4}=-\log 4$; halved → $\log \frac{1}{2}$.

1. Substitute $D^{\ast }=\frac{1}{2}$. Both objective terms become $\log \frac{1}{2}$. Using the expectation fact from the top ($\mathbb{E}$ of a constant is that constant), the sum is $\log \frac{1}{2}+\log (1-\frac{1}{2})=\log \frac{1}{2}+\log \frac{1}{2}.$ Why this step? At equilibrium (cell C) $D^{\ast }=\frac{1}{2}$ everywhere, so each averaged term is just the constant $\log \frac{1}{2}$.

2. Add the logs (the classic form). $\log \frac{1}{2}+\log \frac{1}{2}=\log \left(\frac{1}{2}\cdot \frac{1}{2}\right)=\log \frac{1}{4}=-\log 4.$ Why this step? Log turns the product into a sum (the parent's reason for using log); the famous textbook constant is this summed value.

3. Reconcile with this page's $V$. Our $V$ carries a $\frac{1}{2}$, so our equilibrium value is $\frac{1}{2}(-\log 4)=\log \frac{1}{2}\approx -0.6931$ — exactly the floor used in Ex 3 and Ex 7. The textbook "$-\log 4$" and our "$-0.6931$" differ only by the $\frac{1}{2}$ averaging factor. Why this step? This ties the two conventions together so nothing looks contradictory across the page.

Verify: summed $2\log (0.5)=\log (0.25)=-\log 4\approx -1.3863$; halved $=\log (0.5)\approx -0.6931$. ✓

---

Cell I — Mode collapse limiting case

Forecast: at the abandoned mode $x=-3$, D should be certain it's real ($D^{\ast }=1$); at the over-produced mode, D leans fake.

1. At $x=-3$ (abandoned mode). $D^{\ast }(-3)=\frac{p_{data}(-3)}{p_{data}(-3)+p_g(-3)}=\frac{0.5}{0.5+0}=1.$ Why this step? Same degenerate-zero logic as cell E — G never visits here, so any sample here is surely real.

2. At $x=+3$ (over-produced mode). $D^{\ast }(+3)=\frac{p_{data}(+3)}{p_{data}(+3)+p_g(+3)}=\frac{0.5}{0.5+1.0}=\frac{0.5}{1.5}=\frac{1}{3}\approx \mathrm{0.3333.}$ Why this step? Here G floods the region, so real is outnumbered $1:2$ → D leans "fake", giving only $\frac{1}{3}$ probability of "real".

3. Quantify the miss. A healthy GAN would have $D^{\ast }=\frac{1}{2}$ everywhere. We measure how far the collapsed generator drags D from that ideal by summing the two deviations: $\text{collapse score}=|D^{\ast }(-3)-\frac{1}{2}|+|D^{\ast }(+3)-\frac{1}{2}|=|1-0.5|+|\frac{1}{3}-\frac{1}{2}|=0.5+0.1667=\mathrm{0.6667.}$ Why this step? A single scalar lets you compare training runs; it is $0$ only at true equilibrium and grows as collapse worsens.

4. Interpret and tie up. The score $0.6667$ (versus $0$ for a healthy GAN) says the generator has drifted badly from equilibrium: it is certain-real at one mode and outnumbered at the other. In real training this shows up as D easily flagging the abandoned mode ($D^{\ast }\to 1$ there) while G keeps flooding a single mode — the diagnostic signature of mode collapse. A fix like the Wasserstein GAN changes the objective to keep gradients meaningful even in these lopsided regions.

Verify: $\frac{0.5}{0.5}=1$, $\frac{0.5}{1.5}=\frac{1}{3}\approx 0.3333$, and $0.5+\left|\frac{1}{3}-\frac{1}{2}\right|=0.6667$. ✓

---

Recall drills

Recall Quick self-test

At perfect equilibrium, what does $D^{\ast }(x)$ equal for every $x$? ::: Exactly $\frac{1}{2}$ — D cannot tell real from fake. Why is the original G-loss dangerous early in training? ::: When $D(G(z))\approx 0$, $\log (1-D(G(z)))\approx 0$ is flat, so the gradient vanishes. What is the summed value of the game at equilibrium, and its $\frac{1}{2}$-averaged form? ::: Summed $-\log 4\approx -1.386$; averaged $\log \frac{1}{2}\approx -0.693$. If $p_g(x_0)=0$ but $p_{data}(x_0)>0$, what is $D^{\ast }(x_0)$? ::: $1$ — D is certain such a sample is real. Which G-loss gives stronger gradients when G is failing? ::: The non-saturating loss $-\log D(G(z))$.