GAN training instability and mode collapse

Overview

Training GANs is notoriously difficult because we're trying to balance two networks playing a minimax game. The generator tries to fool the discriminator, while the discriminator tries to catch the generator's fakes. When this balance breaks, we get either training instability (wild oscillations, non-convergence) or mode collapse (generator produces limited variety).

The Minimax Game: Why Balance Matters

where:

• $D(x)$ is the discriminator's probability that $x$ is real
• $G(z)$ generates a fake sample from noise $z$
• The discriminator $D$ tries to maximize $V$ (correctly classify real vs fake)
• The generator $G$ tries to minimize $V$ (fool the discriminator)

Deriving the Training Instability from First Principles

Step 1: What does the discriminator optimize?

For fixed $G$, the discriminator maximizes: $V(D,G) = \mathbb{E}_{x \sim p_{data}}[\log D(x)] + \mathbb{E}_{x \sim p_g}[\log(1 - D(x))]$

where $p_g$ is the generator's distribution.

Taking the functional derivative with respect to $D(x)$ and setting to zero: $\frac{\delta V}{\delta D(x)} = \frac{p_{data}(x)}{D(x)} - \frac{p_g(x)}{1-D(x)} = 0$

Solving for the optimal discriminator: $D^*(x) = \frac{p_{data}(x)}{p_{data}(x) + p_g(x)}$

Why this step? We're finding what discriminator outputs give maximum classification accuracy when both distributions are known. Notice that if $p_{data}(x) = p_g(x)$ everywhere (perfect generator), then $D^*(x) = \frac{1}{2}$ everywhere—the discriminator is reduced to random guessing.

Step 2: What gradient does the generator receive?

The generator optimizes by gradient descent on: $\mathbb{E}_{z \sim p_z}[\log(1 - D(G(z)))]$

Using the chain rule, the gradient w.r.t. generator parameters $\theta$ is: $\nabla_\theta \mathbb{E}_z[\log(1 - D(G(z)))] = \mathbb{E}_z\left[\frac{-1}{1-D(G(z))} \cdot \nabla_x D(G(z)) \cdot \nabla_\theta G(z)\right]$

Why this step? We need the gradient to update generator parameters $\theta$. Notice there are two factors that depend on the discriminator: the scalar $\frac{-1}{1-D(G(z))}$ and the term $\nabla_x D(G(z))$ (how the discriminator's output changes as we move the sample).

Step 3: The vanishing gradient problem (corrected)

When the discriminator is near-perfect, $D(G(z)) \approx 0$ for fake samples. Let's examine both factors:

1. The scalar factor: $\dfrac{-1}{1-D(G(z))} \approx \dfrac{-1}{1-0} = -1$ — this is a constant, not small, so it does NOT cause the vanishing gradient by itself.

2. The real culprit is $\nabla_x D(G(z))$. A confident, well-trained discriminator has a saturated output: it outputs values very close to $0$ (fake) or $1$ (real) and is flat in between. A flat output means its input-gradient $\nabla_x D \to 0$. So even though the scalar is $-1$, the whole product

$\underbrace{(-1)}_{\text{constant}} \cdot \underbrace{\nabla_x D(G(z))}_{\to\, 0} \cdot \nabla_\theta G(z) \;\to\; 0$

vanishes. The generator receives almost no learning signal.

Why this step? This corrects a common misconception: the gradient does not vanish because $\frac{-1}{1-D}$ shrinks (it approaches the constant $-1$). It vanishes because a too-confident discriminator saturates, making its input-derivative $\nabla_x D \to 0$. When $D$ wins too hard, $G$ stops improving.

$\max_G \mathbb{E}_{z \sim p_z}[\log D(G(z))]$

Gradient becomes: $\nabla_\theta \mathbb{E}_z[\log D(G(z))] = \mathbb{E}_z\left[\frac{1}{D(G(z))} \cdot \nabla_x D(G(z)) \cdot \nabla_\theta G(z)\right]$

Now when $D(G(z)) \approx 0$, the scalar prefactor $\frac{1}{D(G(z))}$ becomes large ($1/0.001 = 1000$). This large prefactor amplifies the (small) saturated $\nabla_x D$, rescuing the learning signal that the original loss lost. This provides a stronger signal early in training.

Why this works: We inverted the problem—instead of minimizing "probability of being fake," we maximize "probability of being real." The two losses share the same fixed points but have very different gradient magnitudes when $D$ is confident.

Mode Collapse: The Copy-Paste Problem

Deriving Mode Collapse from Game Dynamics

Step 1: The generator's incentive structure

At iteration $t$, suppose the generator has learned to produce sample $x_1$ that fools $D$. The generator's loss for producing $x_1$ is: $L_G(x_1) = -\log D(x_1)$

If $D(x_1) = 0.9$ (discriminator thinks it's real), then $L_G = -\log(0.9) \approx 0.105$ (low loss, good!).

Step 2: What if the generator tries variety?

Now suppose the generator tries a different mode $x_2$ from the true distribution. But $D$ hasn't seen $G$ produce $x_2$ type samples before, so $D(x_2) = 0.3$ (discriminator is suspicious).

$L_G(x_2) = -\log(0.3) \approx 1.204$

The loss is much higher! The generator is penalized for trying new modes.

Why this step? The generator's objective only cares about fooling $D$ right now, not about covering the full distribution.

Step 3: The discriminator can only punish current samples

The discriminator updates based on: $L_D = -\mathbb{E}_{x \sim p_{data}}[\log D(x)] - \mathbb{E}_{z}[\log(1-D(G(z)))]$

If $G$ never produces samples from mode $x_2$, the discriminator never learns to check for that mode. The generator has no incentive to explore it.

Why this step? This is the core of mode collapse—$G$ exploits $D$'s blindness to unseen modes.

Iteration 1-100: Generator learns to produce digit "1" (easiest to generate, nearly straight line). Discriminator learns "1" is suspicious.

Iteration 101-200: Generator switches to producing digit "7" (also simple). Discriminator now focused on detecting "7", forgets about "1".

Iteration 201-300: Generator switches back to "1". Discriminator has drifted, doesn't recognize "1" anymore.

Result: Generator oscillates between 2-3 digit types, never learns the full distribution of 0-9. This is mode collapse.

Why this happens: The generator found that switching between a few modes is easier than covering all modes, and the discriminator can't "remember" all modes simultaneously.

Solution: WGAN uses a critic $f_w(x)$ that outputs a real number (no sigmoid), and optimizes the Wasserstein distance:

$W(p_{data}, p_g) = \sup_{\|f\|_L \leq 1} \mathbb{E}_{x \sim p_{data}}[f(x)] - \mathbb{E}_{x \sim p_g}[f(x)]$

The critic is constrained to be 1-Lipschitz continuous (gradient clipping or gradient penalty).

Generator loss becomes: $L_G = -\mathbb{E}_z[f_w(G(z))]$

Why this works:

1. The critic provides non-vanishing gradients everywhere (no sigmoid saturation)
2. The Wasserstein distance "pulls" the generator distribution toward the data distribution across all modes, not just currently visible ones
3. Training is more stable because the objective correlates with sample quality

Detailed steps:

1. Critic update (multiple steps per generator update): $w \leftarrow w + \alpha \cdot \nabla_w \left[\frac{1}{m}\sum_{i=1}^m f_w(x^{(i)}) - \frac{1}{m}\sum_{i=1}^m f_w(G(z^{(i)}))\right]$ Then clip: $w \leftarrow \text{clip}(w, -c, c)$ or apply gradient penalty

2. Generator update: $\theta \leftarrow \theta - \alpha \cdot \nabla_\theta \left[-\frac{1}{m}\sum_{i=1}^m f_w(G(z^{(i)}))\right]$

Why these steps: We're alternating between making the critic better at distinguishing (step 1) and making the generator better at fooling (step 2), but with continuous gradients throughout.

Common Training Instabilities

Why it feels right: "Both networks are equally important, so equal learning rates seem fair."

Why it's wrong: The discriminator's task (binary classification on already-generated samples) is easier than the generator's task (creating realistic samples from noise). With equal learning rates, $D$ typically learns much faster and becomes too strong.

The fix: Use a lower learning rate for $D$ than for $G$, or train $D$ fewer times per $G$ update. Common ratio: train $D$ once per 1-3 $G$ updates (or use TTUR: Two Time-scale Update Rule with different learning rates for D and G in Adam).

Why it feels right: "When $D(G(z)) \to 0$, surely some term in the gradient shrinks to zero and kills learning."

Why it's wrong: The scalar $\frac{-1}{1-D(G(z))} \to -1$ is a constant, not small. The real cause is that a too-confident discriminator saturates, so its input-gradient $\nabla_x D(G(z)) \to 0$. The product vanishes because of the saturated derivative, not the scalar prefactor.

The fix: Use the non-saturating loss $\log D(G(z))$ (whose prefactor $\frac{1}{D}$ blows up to counteract the small $\nabla_x D$), or switch to WGAN whose critic never saturates.

Why it feels right: "Batch norm stabilizes training in other deep networks, so it should help GANs."

Why it's wrong: Batch norm creates dependencies between samples in a batch. In the discriminator, this lets it detect fake batches (all fakes are correlated through BN statistics) vs real batches.

The fix:

• Use Virtual Batch Normalization (VBN): normalize each sample using statistics from a fixed reference batch
• Use Spectral Normalization instead: controls Lipschitz constant of layers without batch dependencies
• Use Layer Normalization or Instance Normalization in generator

Advanced Stabilization Techniques

For a weight matrix $W$ in layer $l$, divide by its spectral norm (largest singular value): $\bar{W} = \frac{W}{\sigma(W)}$

where $\sigma(W) = \max_{\|h\|=1} \|Wh\|_2$ is the spectral norm.

Why this works: This constrains the Lipschitz constant of each layer to be at most 1: $\|f_W(x) - f_W(y)\| \leq \|x - y\|$

The discriminator can't have extremely sharp transitions, which prevents mode collapse and stabilizes training.

How to compute efficiently: Use power iteration method:

1. Initialize random vector $v$
2. Iterate: $u = Wv / \|Wv\|$, $v = W^T u / \|W^T u\|$
3. After convergence: $\sigma(W) \approx u^T W v$

Why this step: Computing exact SVD is $O(n^3)$; power iteration gives us $\sigma(W)$ in $O(n^2)$ per iteration, and we only need 1-2 iterations per training step.

Instead of weight clipping, add a penalty term: $\lambda \mathbb{E}_{\hat{x}}[(\|\nabla_{\hat{x}} f_w(\hat{x})\|_2 - 1)^2]$

where $\hat{x} = \epsilon x + (1-\epsilon)G(z)$ with $\epsilon \sim U[0,1]$ (points on lines between real and fake samples).

Why this works: The Wasserstein distance requires the critic to be 1-Lipschitz. This penalty enforces that the gradient norm is close to 1, which is the tightest form of the Lipschitz constraint.

Derivation:

1. A 1-Lipschitz function satisfies $|f(x) - f(y)| \leq \|x - y\|$
2. For differentiable $f$, this means $\|\nabla f(x)\| \leq 1$ everywhere
3. The optimal critic has $\|\nabla f(x)\| = 1$ almost everywhere
4. So we penalize deviation from norm = 1

Why interpolated points: The region between real and fake samples is where the critic must distinguish most sharply, so that's where we enforce the constraint.

Problem: Training high-resolution GANs (1024×1024) directly is unstable—too much detail for early training.

Solution: Start with low resolution (4×4), gradually add layers to increase resolution (8×8, 16×16, ..., 1024×1024). Each resolution trains until stable before growing.

Implementation:

1. Start Generator outputs 4×4, Discriminator inputs 4×4
2. Add layers: Smoothly fade in new layers using $\alpha \in [0,1]$: $\text{output} = \alpha \cdot \text{new\_layer}(x) + (1-\alpha) \cdot \text{upsample}(\text{old\_output})$
3. Increase $\alpha$ from 0 to 1 over several thousand iterations
4. Stabilize at $\alpha=1$, then add next resolution layer

Why this works: Low-resolution images have less information, so the GAN learns the broad structure first (shape, layout) before details (texture, fine edges). This hierarchical learning is more stable than learning all scales simultaneously.

Monitoring and Diagnosing GAN Training

Q: What are signs that the discriminator is too strong? A: Generator loss stays high and constant, $D(G(z)) \approx 0$ for all generated samples, generated samples don't improve visually

Q: What are signs of mode collapse? A: Low diversity in generated samples (e.g., all faces have same pose/expression), generator loss oscillates but samples don't improve in variety, Inception Score or FID suddenly drops

Q: How can you distinguish oscillation from convergence? A: Plot loss curves over time—convergence shows stabilizing losses, oscillation shows periodic swings. Check sample diversity over time—oscillation often means samples cycle between a few types.

Connections

• 4.5.01-Generative-Adversarial-Networks-fundamentals - Core GAN setup and original objective
• 4.5.07-Wasserstein-GAN-and-improved-training - WGAN solution to instability
• 4.5.09-StyleGAN-and-progressive-growing - Progressive training for high-res stability
• 4.5.10-Conditional-GANs-and-control - Mode collapse in conditional generation
• 3.4.05-Optimization-challenges-in-deep-learning - General optimization issues relevant to GANs
• 4.3.06-Batch-normalization-and-alternatives - Why batch norm causes GAN issues

Flashcards

#flashcards/ai-ml

Imagine you have two friends playing a game. One friend (the Gener

Concept Map

Hinglish (regional understanding)

Hinglish (regional understanding)