Level 2 — RecallGenerative Models

Generative Models

⏱ 30 minutes40 marksprintable — key stays hidden on paper

Time limit: 30 minutes Total marks: 40

Answer all questions. Use $...$ / $...$ notation for mathematics where needed.

Q1. State the fundamental difference between generative and discriminative models in terms of the probability distribution each one learns. Give one example of each. (4 marks)

Q2. Define an autoencoder. Name its two main components and state the training objective (loss) used for a standard (deterministic) autoencoder on continuous data. (4 marks)

Q3. Write the Evidence Lower Bound (ELBO) objective for a Variational Autoencoder and identify the two terms, stating the intuitive role of each. (5 marks)

Q4. Explain the reparameterization trick used in VAEs. Write the sampling equation for a Gaussian latent variable $z$ with mean $\mu$ and standard deviation $\sigma$ , and state why the trick is needed. (4 marks)

Q5. For a VAE with a Gaussian posterior $q(z|x)=\mathcal{N}(\mu,\sigma^2)$ and standard normal prior $p(z)=\mathcal{N}(0,1)$ (one latent dimension), write the closed-form KL divergence term. Then compute its value for $\mu=1$ , $\sigma=1$ . (5 marks)

Q6. Write the original GAN minimax value function $V(D,G)$ . Identify the generator's and discriminator's goals in one sentence each. (4 marks)

Q7. Define mode collapse in GAN training and give one commonly used technique/architecture that mitigates training instability. (4 marks)

Q8. In a DDPM forward diffusion process, the single-step transition is $q(x_t|x_{t-1})=\mathcal{N}(x_t;\sqrt{1-\beta_t}\,x_{t-1},\beta_t I)$ . Using $\alpha_t=1-\beta_t$ and $\bar\alpha_t=\prod_{s=1}^{t}\alpha_s$ , write the closed-form for $q(x_t|x_0)$ . (5 marks)

Q9. Briefly explain the FID (Fréchet Inception Distance) metric: what it measures and whether lower or higher is better. (3 marks)

Q10. In one line each, state what "latent diffusion" (Stable Diffusion) and "classifier-free guidance" contribute to modern text-to-image generation. (2 marks)

Answer keyMark scheme & solutions

Q1. (4 marks)

Generative models learn the joint distribution $p(x,y)$ (or $p(x)$ ), i.e. how the data is generated. (1)
Discriminative models learn the conditional/decision boundary $p(y|x)$ directly. (1)
Example generative: Naive Bayes / GMM / VAE / GAN. (1)
Example discriminative: Logistic Regression / SVM / standard neural classifier. (1)

Q2. (4 marks)

Autoencoder: a neural network trained to reconstruct its input by passing it through a lower-dimensional bottleneck. (1)
Components: encoder ( $x \to z$ ) and decoder ( $z \to \hat{x}$ ). (2, 1 each)
Loss: reconstruction error, typically MSE $\;\mathcal{L}=\|x-\hat{x}\|^2$ . (1)

Q3. (5 marks) $\mathcal{L}_{ELBO}=\mathbb{E}_{q(z|x)}[\log p(x|z)] - D_{KL}\big(q(z|x)\,\|\,p(z)\big)$

Correct expression. (2)
Term 1 = reconstruction likelihood; encourages decoder to reconstruct $x$ . (1.5)
Term 2 = KL regularizer; keeps approximate posterior close to prior. (1.5)

Q4. (4 marks)

Trick: express the stochastic sample as a deterministic function of parameters plus independent noise. (1)
Equation: $z=\mu+\sigma\odot\epsilon,\quad \epsilon\sim\mathcal{N}(0,I)$ . (2)
Why: makes sampling differentiable w.r.t. $\mu,\sigma$ so gradients can flow (backprop through a random node). (1)

Q5. (5 marks) Closed form (1-D): $D_{KL}=\tfrac{1}{2}\big(\mu^2+\sigma^2-1-\ln\sigma^2\big)$

Correct formula. (3)
Substitute $\mu=1,\sigma=1$ : $\tfrac12(1+1-1-\ln 1)=\tfrac12(1)=0.5$ . (2)

Q6. (4 marks) $\min_G\max_D V(D,G)=\mathbb{E}_{x\sim p_{data}}[\log D(x)]+\mathbb{E}_{z\sim p_z}[\log(1-D(G(z)))]$

Correct value function. (2)
Discriminator: maximize ability to distinguish real from fake. (1)
Generator: minimize this / fool the discriminator (make fakes look real). (1)

Q7. (4 marks)

Mode collapse: generator produces limited variety / maps many inputs to few outputs, failing to cover all modes of the data distribution. (2)
Mitigation: WGAN / Wasserstein loss, gradient penalty, minibatch discrimination, spectral normalization, feature matching (any one). (2)

Q8. (5 marks) $q(x_t|x_0)=\mathcal{N}\big(x_t;\sqrt{\bar\alpha_t}\,x_0,\;(1-\bar\alpha_t)I\big)$ Equivalently $x_t=\sqrt{\bar\alpha_t}\,x_0+\sqrt{1-\bar\alpha_t}\,\epsilon$ , $\epsilon\sim\mathcal{N}(0,I)$ .

Correct mean $\sqrt{\bar\alpha_t}x_0$ . (2)
Correct variance $(1-\bar\alpha_t)I$ . (2)
Sampling form. (1)

Q9. (3 marks)

FID measures the distance between distributions of Inception-network features of real and generated images, modeled as Gaussians. (2)
Lower FID = better (closer distributions). (1)

Q10. (2 marks)

Latent diffusion: runs the diffusion process in a compressed latent space (from an autoencoder), reducing compute. (1)
Classifier-free guidance: interpolates conditional and unconditional predictions to strengthen adherence to the prompt without a separate classifier. (1)

[
  {"claim":"VAE KL for mu=1,sigma=1 equals 0.5","code":"mu,sig=1,1\nkl=Rational(1,2)*(mu**2+sig**2-1-log(sig**2))\nresult=(simplify(kl)==Rational(1,2))"},
  {"claim":"KL is zero when mu=0,sigma=1 (posterior=prior)","code":"mu,sig=0,1\nkl=Rational(1,2)*(mu**2+sig**2-1-log(sig**2))\nresult=(simplify(kl)==0)"},
  {"claim":"q(x_t|x0) variance coefficient is 1-alphabar consistent with mean coeff squared plus var = 1","code":"ab=symbols('ab',positive=True)\nmean_coeff_sq=ab\nvar_coeff=1-ab\nresult=simplify(mean_coeff_sq+var_coeff-1)==0"}
]