Generator and discriminator dynamics

The Two-Player Game Setup

What are we optimizing?

GANs solve a minimax optimization problem. We're simultaneously:

Maximizing D's ability to classify real vs fake (D wants to be right)
Minimizing G's detectability (G wants to fool D)

Why this form? Let's derive it from first principles:

What does D do? D is a binary classifier outputing probability that input is real.
- For real data $x$ : D should output close to 1, so we want $\log D(x)$ large (log of high probability)
- For fake data $G(z)$ : D should output close to 0, so we want $\log(1 - D(G(z)))$ large (log of high probability it's fake)
Why expectations? We average over all possible real samples from $p_{data}(x)$ and all possible noise vectors $z$ from prior $p_z(z)$ .
Why log? Logarithm converts products to sums (easier optimization) and provides numerical stability. It's the natural choice from maximum likelihood estimation:
- D's task: maximize likelihood of correct classification
- MLE with binary outcomes → binary cross-entropy → log probabilities

Training Dynamics: The Alternating Dance

Figure — Generator and discriminator dynamics

Phase 1: D Optimization (k steps)

For k iterations (typically k=1 or k=5):

Sample mini-batch of m real examples $\{x^{(1)}, \ldots, x^{(m)}\}$ from $p_{data}$
Sample mini-batch of m noise vectors $\{z^{(1)}, \ldots, z^{(m)}\}$ from $p_z$
Generate fakes: $\tilde{x}^{(i)} = G(z^{(i)})$
Update D by ascending gradient:

$\nabla_{\theta_d} \frac{1}{m} \sum_{i=1}^{m} \left[ \log D(x^{(i)}) + \log(1 - D(G(z^{(i)}))) \right]$

Why this step? We're giving D a supervised learning task: "Here are real images (label=1) and fake images (label=0). Learn to tell them apart." We ascend (not descend) because we're maximizing D's objective.

Phase 2: G Optimization (1 step)

Sample new mini-batch of m noise vectors $\{z^{(1)}, \ldots, z^{(m)}\}$
Update G by descending gradient:

$\nabla_{\theta_g} \frac{1}{m} \sum_{i=1}^{m} \log(1 - D(G(z^{(i)})))$

Why this step? G wants D to fail. When $D(G(z)) \to 1$ , that means D thinks the fake is real, so $\log(1 - D(G(z))) \to -\infty$ , giving G low (good) loss.

Worked example Numerical Example: Early Training

Scenario: G produces terrible fakes, D is strong.

Real image $x$ : D outputs $D(x) = 0.95$
Fake image $G(z)$ : D outputs $D(G(z)) = 0.15$ D's loss (negative of objective we maximize): $-[\log(0.95) + \log(1 - 0.15)] = -[\log(0.95) + \log(0.85)]$ $= -[-0.051 + (-0.163)] = -(-0.214) = 0.214$ D is doing well (low loss for a maximization problem → high objective value).

G's loss: $\log(1 - 0.15) = \log(0.85) = -0.163$ G is doing poorly (high magnitude, wants to increase $D(G(z))$ ).

Next iteration: G updates to make fakes more realistic. Suppose now $D(G(z)) = 0.40$ .

G's new loss: $\log(1 - 0.40) = \log(0.60) = -0.511$ Worse! (More negative). Why?

Why this step fails? In early training, $D(G(z))$ is close to 0. The gradient $\nabla_G \log(1 - D(G(z)))$ becomes tiny (vanishing gradient). G barely learns.

Common mistake The Vanishing Gradient Problem

Wrong approach: Use $\min_G \mathbb{E}_z[\log(1 - D(G(z)))]$ throughout training.

Why it feels right: This is the theoretically correct minimax objective. It's what the math says.

Why it fails: When D is strong (early training), $D(G(z)) \approx 0$ , so: $\log(1 - D(G(z))) \approx \log(1) = 0$ The derivative: $\frac{\partial}{\partial G} \log(1 - D(G(z))) = \frac{-1}{1 - D(G(z))} \cdot D'(G(z)) \cdot G'(z)$

When $D(G(z)) \approx 0$ , the $\frac{-1}{1-0} = -1$ term is small magnitude, and $D'(G(z))$ is also small (D is saturated in confidence).

The fix: Use an alternative objective for G (practical, not theory): $\max_G \mathbb{E}_z[\log D(G(z))]$

Instead of minimizing log-probability of D being right, maximize log-probability of D being wrong. Same Nash equilibrium, better gradients early on.

Formula Non-Saturating Generator Loss

$\mathcal{L}_G = -\mathbb{E}_{z \sim p_z}[\log D(G(z))]$

Gradient comparison:

Original: $\frac{\partial}{\partial G} \log(1 - D(G(z))) = \frac{-D'(G(z)) \cdot G'(z)}{1 - D(G(z)}$
Non-saturating: $\frac{\partial}{\partial G} (-\log D(G(z))) = \frac{-D'(G(z)) \cdot G'(z)}{D(G(z))}$

When $D(G(z)) = 0.05$ (D is confident it's fake):

Original gradient: $\frac{-D'(G(z)) \cdot G'(z)}{0.95} \approx -1.05 D'G'$ (small)
Non-saturating: $\frac{-D'(G(z)) \cdot G'(z)}{0.05} \approx -20 D'G'$ (large!)

The non-saturating version provides strong gradients when G is failing, exactly when we need them.

The Nash Equilibrium: Optimal D*

Derivation from first principles:

For fixed G, we want to maximize: $V(D, G) = \int_x p_{data}(x) \log D(x) dx + \int_x p_g(x) \log(1 - D(x)) dx$

(We converted expectations to integrals using definition $\mathbb{E}[f(x)] = \int f(x)p(x)dx$ )

Combine integrals: $V(D, G) = \int_x \left[ p_{data}(x) \log D(x) + p_g(x) \log(1 - D(x)) \right] dx$

To maximize, take derivative w.r.t. $D(x)$ and set to zero: $\frac{\partial}{\partial D(x)} \left[ p_{data}(x) \log D(x) + p_g(x) \log(1 - D(x)) \right] = 0$

$\frac{p_{data}(x)}{D(x)} - \frac{p_g(x)}{1 - D(x)} = 0$

Cross-multiply: $p_{data}(x) (1 - D(x)) = p_g(x) D(x)$ $p_{data}(x) = D(x)[p_{data}(x) + p_g(x)]$

Therefore: $D^*(x) = \frac{p_{data}(x)}{p_{data}(x) + p_g(x)}$

What does this mean? D outputs the Bayes-optimal probability. It's the ratio of "how likely is this from real data" to "how likely is this from either source".

At perfect equilibrium: $p_g = p_{data}$ , so $D^*(x) = \frac{1}{2}$ everywhere. The discriminator cannot tell real from fake!

Worked example Optimal D Calculation

Scenario: 1D data. Real data $p_{data}(x) = \mathcal{N}(0, 1)$ . Generator produces $p_g(x) = \mathcal{N}(2, 1)$ . At $x = 0$ : $p_{data}(0) = \frac{1}{\sqrt{2\pi}} e^{0} \approx 0.399$ $p_g(0) = \frac{1}{\sqrt{2\pi}} e^{-2} \approx 0.054$

$D^*(0) = \frac{0.399}{0.399 + 0.054} = \frac{0.399}{0.453} \approx 0.88$

At $x = 2$ : $p_{data}(2) = \frac{1}{\sqrt{2\pi}} e^{-2} \approx 0.054$ $p_g(2) = \frac{1}{\sqrt{2\pi}} e^{0} \approx 0.399$

$D^*(2) = \frac{0.054}{0.054 + 0.399} = \frac{0.054}{0.453} \approx 0.12$

Why this makes sense: At $x=0$ (center of real distribution), D is confident it's real (0.88). At $x=2$ (center of fake distribution), D is confident it's fake (0.12).

Training Stability and Collapse Modes

Mode Collapse

What happens: G discovers that producing a single type of output fools D, so it only produces that one mode, ignoring diversity in real data.

Why it occurs: G's objective is to fool D on expectation, not to cover all modes. If one mode always fools D, that's a local minimum.

Dynamics:

G produces samples from multiple modes
D learns to classify most modes correctly
G notices one mode still fools D well
G shifts all capacity to that mode
D eventually learns that mode
G switches to another mode (mode hopping)

Mathematical cause: The objective $\mathbb{E}_z[\log D(G(z))]$ averages over z. If D is weak at one region, G can exploit it by mapping many z values to that region.

Oscillation and Non-Convergence

What happens: D and G chase each other without converging. Loss oscillates.

Why it occurs: The minimax game may not have good gradient dynamics. When D gets too strong, G's gradients vanish. When G gets too good, D's task becomes impossible, gradients explode.

Visualization: Imagine two players running around a circular track, each chasing the other's tail. Neither catches up, they just loop forever.

Worked example Training Dynamics Timeline

Epoch 1-10: D dominates. $D(G(z)) < 0.1$ for all samples. G barely learns (vanishing gradients). Loss: $D \approx -0.1$ , $G \approx -2.3$ .

Epoch 11-30: G learns basic structure. $D(G(z))$ rises to 0.3-0.4. D must work harder. Loss: $D \approx -0.5$ , $G \approx -0.9$ .

Epoch 31-50: Balanced training. $D(G(z)) \approx 0.5$ on average. Both networks improve. Loss: $D \approx -0.69$ , $G \approx -0.69$ .

Epoch 51+: Near equilibrium or mode collapse. Either $D(x) \approx D(G(z)) \approx 0.5$ (success) or G produces one type of output repeatedly (collapse).

Why the losses? At equilibrium, $D(x) = D(G(z)) = 0.5$ : $V(D, G) = \log(0.5) + \log(0.5) = -\log(2) \approx -0.69$

Practical Training Considerations

Choosing k (D steps per G step)

k = 1: Balanced training. D and G improve together. Risk: D might not stay strong enough to provide good gradients.
k > 1 (e.g., k=5): Keep D ahead of G. Ensures strong gradients for G. Risk: D might become too strong, causing vanishing G gradients.
Adaptive k: Increase k if D loss spikes (G is winning), decrease if G loss spikes (D is winning).

Why it matters: The quality of G's gradient depends on D's strength. Too weak D → G learns wrong things. Too strong D → G doesn't learn at all.

Learning Rates

Typical setup: $\alpha_D = 0.002$ , $\alpha_G = 0.0001$ (D learns slightly faster).

Why asymetric? D's task (classification) is easier than G's task (generation). If they learn at the same rate, D dominates.

Batch Normalization Pitfall

Issue: Batch norm in D creates correlation between samples in a batch. If entire batch is real or entire batch is fake, D learns to exploit batch statistics rather than individual sample features.

Solution: Use separate batches for real and fake samples when training D. Apply batch norm within each batch type separately.

Connections

4.5.01-Introduction-to-Generative-Adversarial-Networks: This expands the two-player game concept
4.5.06-GAN-training-algorithm: The practical implementation of these dynamics
4.5.08-Mode-collapse-in-GANs: Deep dive into the collapse failure mode
4.5.09-Waserstein-GAN: Alternative formulation that stabilizes training
3.2.04-Binary-cross-entropy-loss: D's loss function is exactly this
3.4.05-Vanishing-gradients: Why non-saturating loss is necessary
5.3.02-Nash-equilibrium: Game theory foundation of GAN equilibrium

Recall Explain to a 12-Year-Old

Imagine you're trying to learn to draw realistic portraits, and your art teacher is checking your work.

At first, your drawings are terrible - stick figures with no detail. Your teacher instantly knows they're fake. She tells you: "Add shading, fix the proportions." You improve a little.

After practicing, your drawings get better. Now your teacher has to look more carefully. She might say: "The eyes are still wrong, and real people don't have perfectly smooth skin." You learn to add more detail.

This back-and-forth continues. You (the Generator) keep trying to fool your teacher. She (the Discriminator) keeps getting better at spotting mistakes. Each round, both of you improve.

Eventually, you get so good that even your expert teacher can only guess if a drawing is yours or a real photo - she's right only 50% of the time, like flipping a coin! That's when you've "won" the game. But sometimes, you might discover that drawing one specific pose always fools your teacher. So you only draw that one pose over and over. That's boring and not the goal - you should be able to draw many different things. That's called "mode collapse."

The key is: you need your teacher to be good (but not impossible to fool) so she can give you useful feedback on how to improve!

#flashcards/ai-ml

What are the two roles in a GAN and what does each one optimize? :: The Generator (G) tries to minimize the objective function by creating realistic fakes. The Discriminator (D) tries to maximize the objective function by correctly classifying real vs fake samples.

What is the GAN minimax objective function?

\min_G \max_D V(D,G) = \mathbb{E}_{x \sim p_{data}}[\log D(x)] + \mathbb{E}_{z \sim p_z}[\log(1-D(G(z)))]

where D maximizes and G minimizes this value function.

Why does the Generator ignore the first term of the GAN objective?

The term

\mathbb{E}_{x \sim p_{data}}[\log D(x)]

depends only on real data and D's parameters, not on G's parameters. It's a constant with respect to G, so it doesn't affect G's gradient.

What is the vanishing gradient problem in GAN training?

Early in training, D easily spots fakes so

D(G(z)) \approx 0

. The gradient

\nabla_G \log(1-D(G(z)))

becomes very small, causing G to learn extremely slowly or not at all.

What is the non-saturating Generator loss and why is it used?

\max_G \mathbb{E}_z[\log D(G(z))]

instead of

\min_G \mathbb{E}_z[\log(1-D(G(z))]

. It provides stronger gradients when D easily identifies fakes (when G needs the most guidance), solving the vanishing gradient problem.

What is the optimal discriminator formula for a fixed generator?

D^*(x) = \frac{p_{data}(x)}{p_{data}(x) + p_g(x)}

which represents the Bayes-optimal probability that a sample comes from real data rather than the generator.

At GAN equilibrium when $p_g = p_{data}$ , what is $D^*(x)$ ? :: $D^*(x) = 0.5$ for all x. The discriminator cannot distinguish real from fake samples since they come from the same distribution, effectively guessing randomly.

What is mode collapse in GANs?

When the Generator discovers one or few outputs that fool the Discriminator and produces only those, ignoring the diversity of the real data distribution. The Generator has collapsed to generating limited modes.

Why do we typically train D for k steps before each G step?

To keep the Discriminator strong enough to provide meaningful gradients to the Generator. If D becomes too weak, it gives poor feedback. If k=1, D might not maintain sufficient strength.

What is the typical relationship between D and G learning rates?

The Discriminator learning rate is typically slightly higher (e.g., 0.0002) than the Generator learning rate (e.g., 0.0001) because the classification task is easier than generation, preventing D from dominating too strongly.

Concept Map

Hinglish (regional understanding)

Intuition Hinglish mein samjho

GAN training ek interesting game hai jaise counterfeiter aur detective ka match. Generator (G) fake data banata hai jo real lagey, aur Discriminator (D) real aur fake ko identify karta hai. Dono ek-dusre ko improve karne mein help karte hain - jaise D better detection karta hai, toh G ko aur realistic fakes banana padta hai, aur jaise G improve hota hai, D ko aur sharp detector banna padta hai.

Iska objective function ek minimax game hai: D maximize karna chahta hai ki woh kitna sahi classify kar raha hai, jabki G minimize karna chahta hai ki D usko kitna easily pakad leta hai. Mathematically, yeh $\min_G \max_D V(D,G)$ hai, jahan V mein real data pe D ki confidence aur fake data pe D ki rejection dono hai. Training mein pehle D ko k steps train karte hain (usually k=1 ya 5), phir G ko ek step train karte hain. Yeh alternating process tab tak chalta hai jab tak perfect equilibrium nahi mil jata - jab G itna acha ho jata hai ki D sirf guess kar sakta hai (50% accuracy).

Lekin practical training mein kai problems ate hain. Early training mein "vanishing gradient" problem hoti hai - jab D bahut strong ho aur G ke fakes ko easily spot kar le, toh G ko koi gradient nahi milta improve karne ke liye. Isko solve karne ke liye hum "non-saturating loss" use karte hain jo G ko strong gradients deta hai jab woh fail ho raha ho. Dusri badi problem "mode collapse" hai - jab G discover kar leta hai ki ek specific type ka output hamesha D ko fool kar deta hai, toh woh sirf wahi ek chez generate karta rehta hai, diversity khatam ho jati hai.

Theory mein optimal state woh hai jab p_g (generator ka distribution) exactly p_data (real data distribution) ke equal ho jaye. Us time pe D* koi fark nahi pata chalta aur woh har sample ke liye 0.5 probability output karta hai. Real-world mein yeh perfect equilibrium achieve karna mushkil hai, isliye careful tuning chahiye - learning rates adjust karna, k value sahi rakhna, aur batch normalization sahi tarike se use karna zaroori hai stability ke liye.

The Two-Player Game Setup

What are we optimizing?

Training Dynamics: The Alternating Dance

Phase 1: D Optimization (k steps)

Phase 2: G Optimization (1 step)

The Nash Equilibrium: Optimal D*

Training Stability and Collapse Modes

Mode Collapse

Oscillation and Non-Convergence

Practical Training Considerations

Choosing k (D steps per G step)

Learning Rates

Batch Normalization Pitfall

Connections

Concept Map

Hinglish (regional understanding)

Go deeper — visual, from zero

Test yourself — Generative Models

Connections