Worked examples — Latent diffusion (Stable Diffusion)

This is the "run the numbers" companion to Latent diffusion (Stable Diffusion). The parent note told you why latent diffusion works. Here we grind through every kind of calculation it can throw at you — compression ratios, noise schedules, the reverse-step arithmetic, attention costs, and the degenerate edge cases where formulas quietly break.

Before we compute anything, one promise: every symbol below was defined in the parent, but I re-anchor each one the moment it appears. If you have never seen ${\bar{\alpha }}_t$ or a softmax before, you will still be able to follow from line one.

---

The scenario matrix

Every problem in this topic falls into one of these case classes. The worked examples below are each tagged with the cell they cover, and together they hit all of them.

#	Case class	What makes it tricky	Covered by
A	Compression sizing — pick $f$, get latent shape	integer division, channel count	Ex 1
B	Dimension / cost ratio — pixel vs latent	linear (conv) vs quadratic (attention) scaling	Ex 2
C	Forward noising, mid schedule ($0<{\bar{\alpha }}_t<1$)	mixing signal and noise weights	Ex 3
D	Forward noising, limiting values (${\bar{\alpha }}_t\to 1$ and $\to 0$)	degenerate: pure signal / pure noise	Ex 4
E	One reverse (denoising) step	the scary rescale-and-subtract formula	Ex 5
F	Reverse step at $t=1$ (last step)	${\sigma }_t\to 0$, no fresh noise added	Ex 6
G	Cross-attention shapes & a softmax	matrix dimensions must line up; softmax sums to 1	Ex 7
H	Classifier-free guidance blend	the $w$ knob, $w=0$ and large-$w$ limits	Ex 8
I	Real-world word problem — VRAM / time budget	translate English into the ratios above	Ex 9
J	Exam twist — "spot the broken assumption"	zero/degenerate input that voids a formula	Ex 10

---

Example 1 — Case A: compression sizing

Forecast: guess the latent height/width and the total element count before reading on.

Step 1 — find latent height and width. $h=\frac{H}{f}=\frac{1024}{8}=128,w=\frac{W}{f}=\frac{1024}{8}=128$ Why this step? "Downsampling by $f$" literally means each side is divided by $f$. The symbol $f$ (a plain number, here 8) is the shrink factor per spatial axis. We divide, not subtract, because it is a ratio of resolutions.

Step 2 — attach the channel count. The latent has $c=4$ channels (learned features, not RGB). So the latent tensor is $z\in {\mathbb{R}}^{128\times 128\times 4}.$ Why this step? Spatial size alone is not the full tensor; every one of the $128\times 128$ grid locations carries a length-$4$ vector.

Step 3 — count elements. $128\times 128\times 4=65,536.$ Why this step? "How many numbers" = product of all three dimensions.

Verify: The input had $1024\times 1024\times 3=3,145,728$ numbers. Ratio $=3,145,728/65,536=48$. That matches the "48× fewer dimensions" the parent quotes — a good sanity check that $f=8,c=4$ was applied correctly.

---

Example 2 — Case B: dimension vs cost ratio (linear vs quadratic)

Forecast: the attention ratio is one of these — $64$, $4096$, or $48$. Guess which.

Step 1 — spatial-location drop. $N_{\text{pix}}=512^2=262,144,N_{\text{lat}}=64^2=4,096,\frac{N_{\text{pix}}}{N_{\text{lat}}}=\frac{262,144}{4,096}=64.$ Why this step? Convolutions touch each spatial location a fixed amount of times, so their cost is linear in $N$. The ratio of costs is just the ratio of $N$: a $64\times$ saving.

Step 2 — attention cost. Self-attention lets every location compare with every other location: $N$ queries $\times$ $N$ keys $=N^2$ comparisons. So $\frac{N_{\text{pix}}^2}{N_{\text{lat}}^2}={\left(\frac{262,144}{4,096}\right)}^2=64^2=4,096.$ Why this step? The $Q{K}^T$ matrix (from the attention formula) is $N\times N$. Its size — and the softmax over it — grows with the square of $N$. That squared scaling is the whole reason latent space is a bigger win for the attention layers than for the convolutions.

Verify: $64<4096$, and $4096=64^2$ exactly. The linear saving ($64\times$) and the quadratic saving ($4096\times$) bracket the real speedup, because a U-Net mixes both layer types — consistent with the parent's "order of magnitude or more."

---

Example 3 — Case C: forward noising, mid-schedule

Forecast: will $z_t$ be closer to $2.0$ or dominated by noise?

Step 1 — recall the forward formula. $z_t=\sqrt{{\bar{\alpha }}_t}{z}_0+\sqrt{1-{\bar{\alpha }}_t}ϵ.$ Why this step? This is the DDPM forward equation the parent stated. ${\bar{\alpha }}_t\in [0,1]$ is the signal-retention dial. The two square roots are weights: $\sqrt{{\bar{\alpha }}_t}$ keeps some of the clean value, $\sqrt{1-{\bar{\alpha }}_t}$ injects noise. They are chosen so the variance stays controlled (see the figure).

Figure s01 — what it shows: the two curves plot the signal weight $\sqrt{{\bar{\alpha }}_t}$ (cyan) and the noise weight $\sqrt{1-{\bar{\alpha }}_t}$ (amber) across the whole dial ${\bar{\alpha }}_t\in [0,1]$. Read left-to-right: at the right edge the cyan curve is high (all signal), at the left edge the amber curve is high (all noise). The dashed white line marks this example's ${\bar{\alpha }}_t=0.36$, where the cyan dot sits at $0.6$ and the amber dot at $0.8$ — exactly the weights we use in Step 2.

Step 2 — compute the two weights. $\sqrt{{\bar{\alpha }}_t}=\sqrt{0.36}=0.6,\sqrt{1-{\bar{\alpha }}_t}=\sqrt{0.64}=\mathrm{0.8.}$ Why this step? $0.36$ was chosen so both roots are clean numbers — it lets us see the blend, $60\%$ signal weight and $80\%$ noise weight. (They don't sum to 1; it's the squares $0.36+0.64=1$ that do, which is what keeps the total variance sensible.)

Step 3 — blend. $z_t=0.6\times 2.0+0.8\times 1.5=1.2+1.2=\mathrm{2.4.}$ Why this step? Just plug the weights into the formula.

Verify: $0.36+0.64=1$ ✓ (variance bookkeeping holds). With signal weight $0.6$ the clean part contributes $1.2$; noise contributes an equal $1.2$, so we are near the halfway point of the schedule — signal and noise comparable, exactly the "mid-schedule" regime.

---

Example 4 — Case D: forward noising, the two limits

Forecast: which one returns the clean value untouched?

Step 1 — case ${\bar{\alpha }}_t=1$. $z_t=\sqrt{1}\cdot 2.0+\sqrt{0}\cdot 1.5=1\cdot 2.0+0=\mathrm{2.0.}$ Why this step? ${\bar{\alpha }}_t=1$ means "keep 100% signal, 0% noise." The noise weight $\sqrt{1-1}=0$ deletes $ϵ$ entirely. So $z_0$ passes through unchanged — the degenerate "no noise yet" case.

Step 2 — case ${\bar{\alpha }}_t=0$. $z_t=\sqrt{0}\cdot 2.0+\sqrt{1}\cdot 1.5=0+1.5=\mathrm{1.5.}$ Why this step? ${\bar{\alpha }}_t=0$ means "keep 0% signal, 100% noise." The signal weight vanishes; $z_t$ is pure noise, carrying zero memory of $z_0=2.0$. This is why generation starts from $z_T\sim \mathcal{N}(0,I)$: at the end of the forward process the original image is gone.

Verify: At ${\bar{\alpha }}_t=1$, $z_t=z_0$ exactly (recoverable). At ${\bar{\alpha }}_t=0$, $z_t=ϵ$ exactly (all noise). The dial behaves monotonically between the two, matching Ex 3's midpoint.

---

Example 5 — Case E: one reverse (denoising) step

Forecast: should $z_{t-1}$ move toward the clean value (closer to $2.0$) or away?

Step 1 — write the DDPM reverse step. $z_{t-1}=\frac{1}{\sqrt{{\alpha }_t}}\left[z_t-\frac{1-{\alpha }_t}{\sqrt{1-{\bar{\alpha }}_t}}\widehat{ϵ}\right]+{\sigma }_t\mathbf{z}.$ Why this step? ${\alpha }_t=1-{\beta }_t$ is the per-step signal factor (one timestep), while ${\bar{\alpha }}_t$ is the cumulative one. The bracket subtracts the predicted noise, the $1/\sqrt{{\alpha }_t}$ rescales back up, and ${\sigma }_t\mathbf{z}$ re-injects a little randomness for diversity.

Figure s02 — what it shows: it lays out the reverse step as a left-to-right pipeline of boxes. Box 1 is the noisy input $z_t=2.4$. The first arrow ("peel predicted noise") lands on box 2, where we have subtracted $0.125\cdot \widehat{ϵ}$ to get $2.2125$. The second arrow ("restore scale") multiplies by $1/\sqrt{0.9}$ to reach $2.3322$. The third arrow ("diversity") adds ${\sigma }_t\mathbf{z}$ to reach the final $2.4322$. The italic caption at the bottom ties it back to Ex 3/4/6 so you see the same coordinate travel clean → noised → denoised.

Step 2 — the noise-removal coefficient. $\frac{1-{\alpha }_t}{\sqrt{1-{\bar{\alpha }}_t}}=\frac{1-0.9}{\sqrt{1-0.36}}=\frac{0.1}{\sqrt{0.64}}=\frac{0.1}{0.8}=\mathrm{0.125.}$ Why this step? This coefficient scales how much of the predicted noise to peel off. $1-{\alpha }_t={\beta }_t$ is small (only one step's worth of noise came in), so we remove only a sliver.

Step 3 — subtract inside the bracket. $z_t-0.125\cdot \widehat{ϵ}=2.4-0.125\times 1.5=2.4-0.1875=\mathrm{2.2125.}$ Why this step? The U-Net's job was to estimate the noise $\widehat{ϵ}$ sitting in $z_t$. Multiplying that estimate by the coefficient from Step 2 tells us how much noise this single step is responsible for; subtracting it removes exactly that slice, leaving a slightly cleaner latent. We subtract (not add) because we are undoing the forward noising.

Step 4 — rescale. $\frac{1}{\sqrt{{\alpha }_t}}=\frac{1}{\sqrt{0.9}}=1.05409\dots ,1.05409\times 2.2125=2.33218\dots$ Why this step? Removing noise shrinks the vector's magnitude; the $1/\sqrt{{\alpha }_t}$ factor restores the expected scale.

Step 5 — add stochasticity. $z_{t-1}=2.33218+0.2\times 0.5=2.33218+0.1=\mathrm{2.43218.}$

Verify: In Ex 3 this same coordinate was noised to $2.4$ from clean $z_0=2.0$. One reverse step nudged it to $\approx 2.432$; the deterministic part alone ($2.332$) moved toward $2.0$ relative to the noise-inflated $2.4$ — correct direction. The final wiggle up is the injected ${\sigma }_t\mathbf{z}$ term, as designed.

---

Example 6 — Case F: the final step $t=1$ (${\sigma }_t\to 0$)

Forecast: will the answer equal the deterministic part of Ex 5?

Step 1 — reuse the deterministic bracket from Ex 5. $\frac{1}{\sqrt{{\alpha }_t}}\left[z_t-0.125\widehat{ϵ}\right]=1.05409\times 2.2125=2.33218\dots$ Why this step? Nothing changed except ${\sigma }_t$; the noise-removal and rescale are identical.

Step 2 — drop the stochastic term. $z_0=2.33218+\underset{=0}{\underbrace{0\cdot \mathbf{z}}}=2.33218\dots$ Why this step? At the very end we want a clean, deterministic latent to hand to the VAE decoder $\mathcal{D}$. Any leftover random ${\sigma }_t\mathbf{z}$ would smear the final image, so ${\sigma }_1=0$ by construction. This is the degenerate edge case of the reverse formula.

Verify: $z_0=2.33218$ equals exactly the deterministic core of Ex 5 with the $+0.1$ noise removed. So Ex 5's answer $=z_0+{\sigma }_t\mathbf{z}=2.33218+0.1=2.43218$ ✓ — internally consistent.

---

Example 7 — Case G: cross-attention shapes and a softmax

Forecast: how big is $Q{K}^T$ — is it $4096\times 77$ or $77\times 768$?

Step 1 — line up the shapes. $Q$ comes from the image latent: $Q\in {\mathbb{R}}^{N\times d}={\mathbb{R}}^{4096\times 64}$. $K$ comes from text: $K\in {\mathbb{R}}^{77\times 64}$ (after projecting the 768-dim tokens to $d=64$). Then $Q{K}^T\in {\mathbb{R}}^{4096\times 77}.$ Why this step? Matrix product $Q(K^T)$ has shape (rows of $Q$) $\times$ (rows of $K$) $=4096\times 77$. Each latent location (row) gets a similarity score against each of the 77 text tokens (column). That is literally "which words does this patch attend to."

Step 2 — softmax one row. The softmax of scores $s=[2,0,0]$ is $\text{softmax}(s{)}_i=e^{s_i}/{\sum }_j{e}^{s_j}$. $e^2=7.389056,e^0=1,e^0=1,\text{sum}=\mathrm{9.389056.}$ $\text{weights}=\left[\frac{7.389056}{9.389056},\frac{1}{9.389056},\frac{1}{9.389056}\right]=[0.78699,0.10650,0.10650].$ Why this step? Softmax turns raw scores into a probability distribution over tokens: exponentiate (so larger scores dominate, and everything is positive) then normalize (so it sums to 1). The $1/\sqrt{d}$ in the attention formula would divide scores first; here we softmax the given post-scaling scores directly.

Verify: The three weights sum to $0.78699+0.10650+0.10650=0.99999\approx 1$ ✓ — a valid probability distribution. The highest-score token (the "2") gets the largest weight, as it must.

---

Example 8 — Case H: classifier-free guidance blend and its limits

Forecast: which $w$ ignores the text prompt entirely?

Step 1 — case $w=0$. $\widehat{ϵ}=1.0+0\cdot (1.5-1.0)=1.0={ϵ}_{\varnothing }.$ Why this step? $w=0$ multiplies the "push toward the prompt" term $({ϵ}_c-{ϵ}_{\varnothing })$ by zero, deleting it. What remains is the unconditional prediction ${ϵ}_{\varnothing }$ — the image ignores the text. This is the degenerate no-guidance case.

Step 2 — case $w=1$. $\widehat{ϵ}=1.0+1\cdot (1.5-1.0)=1.0+0.5=1.5={ϵ}_c.$ Why this step? At $w=1$ the blend ${ϵ}_{\varnothing }+1\cdot ({ϵ}_c-{ϵ}_{\varnothing })$ algebraically collapses to ${ϵ}_c$. So $w=1$ is just ordinary text conditioning — no extra amplification, the plain prompt-following prediction.

Step 3 — case $w=7.5$. $\widehat{ϵ}=1.0+7.5\cdot (1.5-1.0)=1.0+7.5\times 0.5=1.0+3.75=\mathrm{4.75.}$ Why this step? $w>1$ pushes past ${ϵ}_c$ along the same direction "away from unconditional, toward the prompt," exaggerating it. This is why the SD default $w=7.5$ gives strongly prompt-faithful (sometimes over-saturated) images: we overshoot the conditioned prediction on purpose.

Verify: As a linear function of $w$, $\widehat{ϵ}(w)=1.0+0.5w$. Check: $\widehat{ϵ}(0)=1.0$, $\widehat{ϵ}(1)=1.5$, $\widehat{ϵ}(7.5)=4.75$ all lie on that line ✓. Monotone increasing in $w$, matching "more guidance = stronger prompt pull."

---

Example 9 — Case I: real-world VRAM / time word problem

Forecast: will the pixel version fit in a 24 GB consumer GPU?

Step 1 — pixel-space attention matrix bytes. $N_{\text{pix}}^2\times 4=262,144^2\times 4=68,719,476,736\times 4=274,877,906,944\text{ bytes}.$ Convert: $274,877,906,944/1024^3=256$ GiB. Why this step? The $Q{K}^T$ matrix is $N\times N$; storing it costs $N^2$ entries $\times 4$ bytes. This is the quadratic blow-up from Ex 2 turned into concrete memory.

Step 2 — latent-space attention matrix bytes. $N_{\text{lat}}^2\times 4=4,096^2\times 4=16,777,216\times 4=67,108,864\text{ bytes}=64\text{ MiB}.$ Why this step? Same formula, smaller $N$. Because the memory grows as $N^2$, a $64\times$ smaller $N$ shrinks the matrix by $64^2$.

Step 3 — reduction factor. $\frac{256\text{ GiB}}{64\text{ MiB}}=\frac{256\times 1024\text{ MiB}}{64\text{ MiB}}=4,096.$ Why this step? Same $N^2/N^2=4096$ from Ex 2 — now interpreted as memory, not FLOPs.

Verify: $256$ GiB $\gg 24$ GB, so pixel-space attention at this resolution cannot fit on a consumer GPU — while $64$ MiB fits trivially. The $4096\times$ reduction is exactly the quadratic ratio of Ex 2 ✓. This is why latent diffusion made high-res text-to-image possible on ordinary hardware.

---

Example 10 — Case J: exam twist — spot the broken assumption

Forecast: guess the one line that becomes impossible.

Step 1 — evaluate the noise weight. $\sqrt{1-{\bar{\alpha }}_t}=\sqrt{1-1.2}=\sqrt{-0.2}.$ Why this step? The square root of a negative number is not a real value — the noise weight is undefined. The forward formula silently assumes $1-{\bar{\alpha }}_t\ge 0$.

Step 2 — derive the valid range. We need both weights real: ${\bar{\alpha }}_t\ge 0$ (so $\sqrt{{\bar{\alpha }}_t}$ is real) and $1-{\bar{\alpha }}_t\ge 0$ (so the noise weight is real). Together: $0\le {\bar{\alpha }}_t\le 1.$ Why this step? Recall ${\bar{\alpha }}_t={\prod }_{i=1}^t(1-{\beta }_i)$ with each ${\beta }_i\in (0,1)$, so every factor is in $(0,1)$ and their product always lands in $[0,1]$. The value $1.2$ proves at least one ${\beta }_i<0$ — an invalid schedule that leaves the $[0,1]$ box.

Verify: At the two legal endpoints ${\bar{\alpha }}_t\in \{0,1\}$ the formula gave clean, finite answers in Ex 4. At ${\bar{\alpha }}_t=1.2$ it produces $\sqrt{-0.2}$, non-real — so the range $[0,1]$ is exactly the safe domain, and the misconfigured schedule is the true bug. The formula was never wrong; the input violated its assumption.

---

Recall Self-test

$f=8$, feed $256\times 256$: latent spatial size? ::: $256/8=32$, so $32\times 32\times 4$. ${\bar{\alpha }}_t=0.36$: signal weight $\sqrt{{\bar{\alpha }}_t}$? ::: $0.6$. ${\bar{\alpha }}_t\to 0$ gives $z_t=$? ::: pure noise $ϵ$ (all signal weight vanishes). Attention cost scales like? ::: $N^2$ (quadratic in spatial locations). What is ${\beta }_t$? ::: the small positive per-step noise-variance fraction; ${\alpha }_t=1-{\beta }_t$. CFG with $w=1$ gives? ::: the plain conditioned prediction ${ϵ}_c$. Valid range of ${\bar{\alpha }}_t$? ::: $[0,1]$; outside it a square root goes non-real.

See also: Denoising Diffusion Probabilistic Models (DDPM) for the forward/reverse formulas, U-Net Architecture for where the attention lives, CLIP Text Encoder and Attention Mechanisms for Ex 7, Variational Autoencoders (VAE) for the encode/decode ends, and DDIM Sampling for the deterministic (${\sigma }_t=0$) variant hinted at in Ex 6.