Generative Adversarial Networks (GAN) framework: Visual walkthrough

This page rebuilds the single most important fact about the GAN framework entirely from pictures: when the counterfeiter becomes perfect, the detective is forced to flip a coin. We will earn every symbol before we use it. A twelve-year-old who has never seen an integral should be able to follow from line one.

Step 0 — What are we even measuring?

WHAT. Before any formula, meet the two characters as piles of data.

WHY. Everything downstream is a comparison of two piles. If you can see the piles, the algebra later is just bookkeeping.

PICTURE. Look at Figure s01. The cyan curve is $p_{data} (x)$ — read it as "how often real examples land at value $x$ ." Tall = common, low = rare. The amber curve is $p_{g} (x)$ — the same idea for the generator's fakes. The horizontal axis $x$ is "data space" (imagine every possible image squashed onto one line).

Figure — Generative Adversarial Networks (GAN) framework

Our entire goal, stated visually: push the amber curve until it sits perfectly on the cyan curve.

Step 1 — The generator, and where its fakes come from

WHAT. Before we can score anything, we must build the machine that makes the fakes: the generator $G$ , fed by a noise distribution $p_{z}$ .

WHY. The value function in Step 2 will contain the symbols $G$ , $z$ and $p_{z}$ . The contract says: never use a symbol before it is defined and pictured. So we define them here, first.

PICTURE. In Figure s02 a random seed $z$ (a little dot picked from the noise cloud on the left) is pushed through the box $G$ and lands as a fake sample $G (z)$ on the $x$ -axis. Different seeds land in different places; collect all their landing spots and you rebuild the amber curve $p_{g}$ from Step 0.

Step 2 — What the discriminator is, drawn as a dial

WHAT. We introduce $D (x)$ — the discriminator — as a dial reading attached to every point $x$ .

WHY. People get lost because $D$ is a network with millions of weights. For this proof, none of that matters. All that matters is: at each $x$ , $D$ outputs one number in $[0, 1]$ . So we draw it as a needle.

PICTURE. In Figure s03, at three sample points the white needle shows $D (x)$ : near the cyan bump it points to $\approx 1$ ("surely real"), near the amber bump it points to $\approx 0$ ("surely fake"), and where the curves overlap it sits mid-scale.

The symbol $D (x)$ is now earned: it is just the needle position at location $x$ .

Step 3 — Scoring the detective: the value function

WHAT. We build $V (G, D)$ , the number that says how well the detective is doing.

WHY. To find the "best" detective we need a scoreboard. Two things must go on it: reward for calling real things real, and reward for calling fakes fake.

PICTURE. Figure s04 splits the scoreboard into two coloured bars. Read the equation left-to-right as the picture is coloured (every symbol below was defined in Steps 0–2):

V (G, D) = reward on reals E_{x \sim p_{data}} [lo g D (x)] + reward on fakes E_{z \sim p_{z}} [lo g (1 - D (G (z)))]

Term by term, right where each symbol lives:

$E_{x \sim p_{data}} [\cdot]$ ::: "average over real samples." $E$ = expectation = weighted average, weighting each $x$ by how common it is under $p_{data}$ .
$lo g D (x)$ ::: the reward when $x$ is real. If $D (x) \to 1$ , $lo g 1 = 0$ (best, no penalty). If $D (x) \to 0$ , $lo g 0 = - \infty$ (catastrophe). So the detective is pushed to raise $D$ on reals.
$E_{z \sim p_{z}} [\cdot]$ ::: "average over noise seeds," drawn from the noise distribution $p_{z}$ we defined in Step 1.
$G (z)$ ::: a fake sample — the generator's landing point for seed $z$ .
$1 - D (G (z))$ ::: probability the detective assigns to "this fake is fake." We want this near $1$ .
$lo g (1 - D (G (z)))$ ::: reward for correctly doubting a fake.

The full game is $min_{G} max_{D} V (G, D)$ : the detective climbs $V$ , the forger drags it down.

Step 4 — Freeze the forger, find the perfect detective

WHAT. Hold $G$ fixed (freeze the amber curve). Ask: what is the best $D$ the detective could possibly pick?

WHY. The minimax is two moving targets — impossible to attack at once. So we do the classic trick: solve the inner problem (best $D$ for a frozen $G$ ) first. This is the same "one variable at a time" idea behind alternating optimisation.

PICTURE. Figure s05 rewrites the fake-reward term so it lives on the same $x$ -axis as the reals. Here is why we may do that. The fake-reward term averages a function of $G (z)$ over seeds $z \sim p_{z}$ . But by the push-forward idea from Step 1, drawing a seed and mapping it through $G$ is the same as drawing a landing point $x$ directly from $p_{g}$ . So the average over seeds equals an average over landing points:

E_{z \sim p_{z}} [lo g (1 - D (G (z)))] = \int_{z} p_{z} (z) lo g (1 - D (G (z))) d z = \int_{x} p_{g} (x) lo g (1 - D (x)) d x

Middle step ::: an expectation is an integral against the density — $E_{z \sim p_{z}} [h (z)] = \int p_{z} (z) h (z) d z$ .
Right step (change of variables $x = G (z)$ ) ::: this is exactly the definition of the push-forward density $p_{g}$ — the amount of $p_{z}$ -mass that lands near $x$ is $p_{g} (x)$ . So every $z$ is relabelled by where it lands, and $p_{z} (z) d z$ becomes $p_{g} (x) d x$ .

The real-reward term is already an integral, $\int_{x} p_{data} (x) lo g D (x) d x$ . Both now sit on the same $x$ -axis, so we combine them:

V (G, D) = \int_{x} [real term p_{data} (x) lo g D (x) + fake term p_{g} (x) lo g (1 - D (x))] d x

$\int_{x} \dots d x$ ::: "add up over every point $x$ ." An integral is just a sum over infinitely many thin slices.
The bracket is a separate little contest at each single $x$ . Nothing at point $x$ talks to point $x^{'}$ . So we can optimise one $x$ at a time — this is the key that unlocks everything.

Step 5 — The one-point tug-of-war

WHAT. Fix a single location $x$ . Let $a = p_{data} (x)$ , $b = p_{g} (x)$ , and let $d = D (x)$ be the dial we get to turn. Maximise

$f (d) = a lo g d + b lo g (1 - d) .$

WHY. By Step 4 the whole integral is maximised by winning every one of these tiny 1-D contests. So the giant functional problem collapses to ordinary calculus in one variable — the kind you can graph.

PICTURE. Figure s06 plots $f (d)$ against $d \in (0, 1)$ for a sample $a$ and $b$ . It is a smooth hill with exactly one peak. The two anchors — $a$ pulling the dial up toward $1$ , $b$ pulling it down toward $0$ — balance at the top.

To find the peak we ask "where is the slope zero?" That is why we take a derivative — the derivative is the tool that answers "which way is uphill, and where does uphill stop?"

\frac{df}{dd} = up-pull from reals \frac{a}{d} - down-pull from fakes \frac{b}{1 - d} = 0

Set the two pulls equal and solve:

\frac{a}{d} = \frac{b}{1 - d} ⟹ a (1 - d) = b d ⟹ a = d (a + b) ⟹ d^{*} = \frac{a}{a + b}

Putting the names back:

Step 6 — Every case of $D^{*}$ , checked by eye

WHAT. Walk through all sign/zero/degenerate cases of $D^{*} = \frac{a}{a + b}$ so no scenario surprises the reader.

WHY. The contract: cover every case. A fraction with sums in the denominator has three interesting regimes and one forbidden one.

PICTURE. Figure s07 colours the $x$ -axis by which regime it is in.

Case	Meaning	$D^{*} (x)$	Picture region
Only reals here ( $b = 0, a > 0$ )	genuine data, no fakes	$\frac{a}{a} = 1$	pure cyan zone
Only fakes here ( $a = 0, b > 0$ )	forger hallucinated a region	$\frac{0}{b} = 0$	pure amber zone
Equal overlap ( $a = b > 0$ )	curves cross	$\frac{a}{2 a} = 0.5$	the shrug line
No data at all ( $a = b = 0$ )	neither curve is here	$\frac{0}{0}$ undefined	white gap

Note $D^{*} (x) \in [0, 1]$ always (a share of a pie can't exceed the whole pie), so it is a valid probability — the definition of $D$ from Step 2 is never violated.

Step 7 — Now let the forger move: the finish line

WHAT. Plug $D^{*}$ back in and ask what the forger's best move does.

WHY. We froze $G$ in Step 4. Un-freeze it: the generator now drags $V$ down, i.e. drives $p_{g}$ toward $p_{data}$ . We show the endpoint.

PICTURE. Figure s08 shows the amber curve sliding until it lies exactly on the cyan curve. When $p_{g} (x) = p_{data} (x)$ everywhere, substitute $b = a$ into $D^{*}$ :

D^{*} (x) = \frac{a}{a + a} = \frac{a}{2 a} = \frac{1}{2} for every x .

The needle from Step 2 collapses to the middle of the dial everywhere. The detective, facing a perfect forger, can do no better than a coin flip. This flat $D \equiv \frac{1}{2}$ is the Nash equilibrium of the game: neither player can improve by moving alone.

The one-picture summary

Figure s09 stacks the whole journey: two piles → a dial → a scoreboard → a one-point hill → its peak $\frac{a}{a + b}$ → curves merging → the flat $0.5$ line.

Recall Feynman retelling — say it to a friend

"Picture two heaps of sand on a number line: real photos in cyan, fake ones in amber. The amber heap is built by a machine $G$ that grabs random dice-throws (the noise $z$ ) and flings each one to a spot on the line — pile up where they land and that's the amber curve. A judge stands over the line holding a dial that reads $0$ ('fake') to $1$ ('real'). We score the judge: full marks for pointing the dial up where cyan sand is, and down where amber sand is. If you freeze the forger and ask 'what dial setting scores best right here?', the answer is embarrassingly simple — it's just what fraction of the sand right here is real, $\frac{real}{real + fake}$ . That's the best any judge can ever do. Now let the forger pile his amber sand exactly on top of the cyan. At every spot, half the sand is real and half is fake, so the best dial setting is $\frac{1}{2}$ — a shrug — everywhere. The forger has won by making the judge helpless, and that helpless flat line at $0.5$ is precisely the goal of training a GAN."

Recall Quick self-test

What is $D^{*} (x)$ in words? ::: The fraction of the probability mass at $x$ that is real: $p_{data} / (p_{data} + p_{g})$ . What is $p_{z}$ and is it learned? ::: The fixed noise distribution we draw seeds $z$ from; it is not learned — only $G$ and $D$ are. Why can we swap the average over $z$ for an integral over $x$ with $p_{g}$ ? ::: Because $p_{g}$ is the push-forward of $p_{z}$ through $G$ : sampling $z$ then mapping through $G$ is the same as sampling $x$ from $p_{g}$ . Why can we optimise one point $x$ at a time? ::: Because $V$ is an integral of independent per-point terms — nothing at $x$ affects the contest at any other $x^{'}$ . What does $D = 0.5$ everywhere mean? ::: $p_{g} = p_{data}$ ; the generator's fakes are indistinguishable from real data — the Nash equilibrium. Where is $D^{*}$ undefined and why is that OK? ::: Where $p_{data} = p_{g} = 0$ ; no sample lands there, so it contributes $0$ to $V$ .