Generator and discriminator dynamics: Visual walkthrough

Step 1 — What is a probability density curve?

WHAT. Before any GAN math, we need one picture: a density curve. Imagine every possible data point $x$ (say, a number on a horizontal line). Over that line we draw a curve. Where the curve is tall, data of that kind is common; where it is low, that kind of data is rare.

WHY this tool. GANs compare two populations of things — real samples and fake samples. To talk about "how common is this exact value in the real set vs the fake set" we need a height-at-each-point function. That is exactly a density. We call the real one $p_{d a t a} (x)$ and the fake one $p_{g} (x)$ .

One thing that makes a density special. A density is not just any height function: the total area under its curve is exactly $1$ , $\int_{x} p_{d a t a} (x) d x = 1, \int_{x} p_{g} (x) d x = 1.$ That $= 1$ says "all the real data, taken together, is 100% of the real data" — the curve simply spreads that one whole unit of certainty across the axis. This normalization is why we may call these curves densities and read heights as relative commonness rather than arbitrary numbers.

PICTURE. Two bell-shaped humps on the same axis, each enclosing area $1$ . The blue hump $p_{d a t a}$ = the real data. The orange hump $p_{g}$ = whatever the Generator currently spits out. The subscript $g$ just means "generated".

Figure — Generator and discriminator dynamics

At each $x$ there are only two numbers that matter: the blue height and the orange height. Everything below flows from comparing them.

Step 2 — What the Discriminator is being asked

WHAT. The Discriminator $D$ is a function that eats a point $x$ and returns one number between 0 and 1. We read it as "the probability that this $x$ came from the real pile."

WHY between 0 and 1. A number you can honestly call a probability lives in $[0, 1]$ : $0$ = "definitely fake", $1$ = "definitely real", $0.5$ = "no idea". Anything outside that range would not be interpretable as a chance.

PICTURE. A short green curve $D (x)$ hovering between the floor (0) and ceiling (1). Its job is to sit high where blue dominates and low where orange dominates.

Recall

If $D (x) = 0.9$ , what is $D$ claiming about that point? ::: "There is a 90% chance this point came from the real data."

Step 3 — The scoring rule: reward for being confidently right

WHAT. We need to score the Discriminator so we know which $D$ is best. The GAN score at a single point $x$ is

$how much real mass sits here p_{d a t a} (x) reward for calling it real lo g D (x) + how much fake mass sits here p_{g} (x) reward for calling it fake lo g (1 - D (x)) .$

WHY the $lo g$ . Look at the shape of $lo g$ : it is $- \infty$ near 0 and climbs slowly toward 0 as its input approaches 1. So $lo g D (x)$ punishes hard when $D$ says "fake" ( $D$ near 0) about a real-heavy point, but gives only a gentle reward for being right. That asymmetry is the pressure that forces honesty. (This is the same binary cross-entropy you have met before, just written with a $+$ sign because we maximize instead of minimize.)

WHY two terms. The first term pays $D$ for shouting "real!" exactly where real mass ( $p_{d a t a}$ ) piles up. The second pays $D$ for shouting "fake!" where fake mass ( $p_{g}$ ) piles up. Weighting each reward by the mass present means points nobody visits barely affect the score.

PICTURE. The curve $lo g$ from $x = 0$ to $x = 1$ , with the steep cliff near 0 marked in red.

Step 4 — Optimise each point on its own

WHAT. Here is the key liberation. Notice $D (x)$ at one point $x$ appears only in that point's bracket — it does not tangle with $D$ at any other point. So maximizing the whole integral = maximizing each bracket separately. We drop the integral and stare at a single point.

WHY we're allowed. If I can make every individual bracket as big as possible, their sum is automatically as big as possible. No point's best choice hurts another's. This is why the problem, which looked infinite-dimensional, collapses to a one-variable puzzle.

PICTURE. Freeze one vertical slice at a chosen $x_{0}$ . Read off the two fixed heights there — call them $a = p_{d a t a} (x_{0})$ (blue) and $b = p_{g} (x_{0})$ (orange). They are just numbers now. The only free knob is $d = D (x_{0})$ , sliding between 0 and 1.

Our one-point puzzle: $f (d) = blue height (fixed) a lo g d + orange height (fixed) b lo g (1 - d), d \in (0, 1) .$

Step 5 — Find the top of the hill (the derivative tool)

WHAT. We want the $d$ that makes $f (d)$ largest. The tool for "top of a smooth hill" is the derivative: it measures the slope. At the very top the slope is flat — zero.

WHY the derivative and not guessing. $f (d)$ rises then falls (we'll confirm the shape), so its peak is the single spot where the tangent line is horizontal. Setting slope $= 0$ pins that spot exactly, no trial and error.

Differentiate term by term. The slope of $lo g d$ is $\frac{1}{d}$ ; the slope of $lo g (1 - d)$ is $\frac{- 1}{1 - d}$ (the minus from the inside $1 - d$ shrinking as $d$ grows):

$f^{'} (d) = pull up from real reward \frac{a}{d} - pull down from fake reward \frac{b}{1 - d} = 0.$

PICTURE. The hill $f (d)$ over $[0, 1]$ , its rounded peak marked, the flat tangent line drawn at the top, and the two competing "pulls" shown as opposing arrows.

Step 6 — Solve the balance equation

WHAT. Set the two pulls equal and untangle $d$ .

$\frac{a}{d} = \frac{b}{1 - d} .$

Cross-multiply (each side's top times the other side's bottom): $a (1 - d) = b d ⟹ a - a d = b d ⟹ a = d (a + b) .$

Divide by the total height $a + b$ : $d^{*} = \frac{a}{a + b} i.e. D^{*} (x) = \frac{p _{d a t a} ( x )}{p _{d a t a} ( x ) + p _{g} ( x )} .$

WHY it makes sense. The best guess at a point is simply the blue share of the total height. If real mass is 3 units and fake mass is 1 unit, $D^{*} = \frac{3}{4}$ — "three out of four things here are real, so bet 0.75 on real." It is pure book-keeping of the two heights.

PICTURE. A stacked bar at $x_{0}$ : blue part $a$ on top of orange part $b$ ; a ruler shows the blue fraction $\frac{a}{a + b}$ equals the green dot $D^{*} (x_{0})$ .

Recall

Why does no other $d$ beat $d^{*} = \frac{a}{a + b}$ ? ::: Because $f^{''} (d) = - \frac{a}{d ^{2}} - \frac{b}{( 1 - d ) ^{2}} < 0$ everywhere, so the hill is strictly concave — one peak, and the flat-slope point is it.

Step 7 — The degenerate corners (the cases the parent skipped)

WHAT. The tidy formula assumed both $a$ and $b$ were positive. What if one is zero? These are real situations: a point the Generator never produces, or a region real data never visits.

Only real mass here ( $a > 0, b = 0$ ): $D^{*} = \frac{a}{a + 0} = 1$ . The green curve pins to the ceiling — "if the Generator literally never makes this, any sighting is certainly real."
Only fake mass here ( $a = 0, b > 0$ ): $D^{*} = \frac{0}{0 + b} = 0$ . Floor. "Real data never comes here, so it's certainly a fake."
Empty region ( $a = 0, b = 0$ ): the point-score is $0 \cdot lo g d + 0 \cdot lo g (1 - d) = 0$ for every $d$ . $D$ is undefined here and it does not matter — nobody, real or fake, ever lands here, so no score is at stake. We may set $D = \frac{1}{2}$ by convention.

WHY handle these. Near the tails of the humps the heights genuinely fall to (numerically) zero. Without these cases the reader hits a $\frac{0}{0}$ and panics; here we've shown it's harmless.

PICTURE. Three mini-panels: real-only tail (green glued to 1), fake-only tail (green glued to 0), and the flat "no-man's-land" gap where $D$ is a free-floating dashed line.

Step 8 — The equilibrium: when the Generator wins

WHAT. Now let the Generator succeed completely, so its curve matches real data: $p_{g} = p_{d a t a}$ , i.e. orange lies exactly on blue. Plug in $b = a$ :

$D^{*} (x) = \frac{a}{a + a} = \frac{a}{2 a} = \frac{1}{2} at every x .$

WHY this is the finish line. A flat $D^{*} \equiv \frac{1}{2}$ means the best-possible Discriminator is reduced to a coin flip everywhere. It cannot do better than guessing — the two piles are indistinguishable. This is the Nash equilibrium of the two-player game and the goal of GAN training. (When the Generator instead cheats by matching only one hump, you get mode collapse; the fix in Wasserstein GAN changes the score in Step 3 to avoid the flat-gradient trap.)

PICTURE. Blue and orange humps perfectly overlapping into one purple curve; the green $D^{*}$ line collapsing to a dead-flat 0.5.

The one-picture summary

The whole derivation on one canvas: two density humps (blue real, orange fake); at any slice the green $D^{*}$ is just the blue fraction of the stacked heights; where they overlap, that fraction is $\frac{1}{2}$ .

Recall Feynman retelling (say it out loud)

Picture two overlapping hills of stuff — one hill is real data, one is the fakes, and each hill holds exactly one whole unit of stuff (area $1$ ). Pick any spot on the ground. Ask: of the stuff piled here, what fraction is real? That fraction — real height divided by (real + fake) height — is the smartest possible guess for "is this real?", and that guess is the optimal Discriminator, $D^{*} = \frac{p _{d a t a}}{p _{d a t a} + p _{g}}$ . Where only real stuff sits, the guess is 1; where only fakes sit, it's 0; where nobody sits, it doesn't matter. And when the Generator gets so good the fake hill lands exactly on the real hill, every spot is half real, half fake — the guess is 0.5 everywhere, a pure coin flip. That flatline is the moment the Generator has won.