Visual walkthrough — Noise scheduling

This is a visual companion to the parent note. If you want the same story in Hinglish, see 4.5.12 Noise scheduling (Hinglish).

---

Step 0 — The vocabulary we are allowed to use

Before any symbol appears in a formula, let us earn it in plain words.

That is our entire alphabet. Everything below is built from just these.

---

Step 1 — One smudge: the single-step rule

WHAT. One step of the forward process takes the current picture and produces a slightly noisier one: $x_t=\underset{\text{shrink signal}}{\underbrace{\sqrt{{\alpha }_t}}}\underset{\text{old picture}}{\underbrace{x_{t-1}}}+\underset{\text{noise volume}}{\underbrace{\sqrt{1-{\alpha }_t}}}\underset{\text{fresh static}}{\underbrace{{ϵ}_{t-1}}}$

WHY these exact square-roots. Why not just ${\alpha }_t{x}_{t-1}+(1-{\alpha }_t)ϵ$? Because we care about variance (spread), and variance grows with the square of a multiplier. If you scale a bell curve of spread 1 by a number $c$, its spread becomes $c^2$. So to make the signal keep fraction ${\alpha }_t$ of its variance and the noise carry fraction $(1-{\alpha }_t)$ of the variance, the multipliers must be $\sqrt{{\alpha }_t}$ and $\sqrt{1-{\alpha }_t}$. Then $(\sqrt{{\alpha }_t}{)}^2+(\sqrt{1-{\alpha }_t}{)}^2={\alpha }_t+1-{\alpha }_t=1$: total spread is conserved at 1. That is the whole reason for the square-roots.

PICTURE. The old picture arrow gets shortened by $\sqrt{{\alpha }_t}$; a perpendicular noise arrow of length $\sqrt{1-{\alpha }_t}$ is added on. Because they are perpendicular (independent), their lengths combine by Pythagoras to give back length 1.

---

Step 2 — Why noise adds like Pythagoras, not like ordinary numbers

WHAT. We need one fact before we can chain steps together. When you add two independent bell-curve noises, their spreads combine like the sides of a right triangle: $\text{if }{Z}_1\sim \mathcal{N}(0,{\sigma }_1^2)\text{and}{Z}_2\sim \mathcal{N}(0,{\sigma }_2^2),Z_1+Z_2\sim \mathcal{N}(0,{\sigma }_1^2+{\sigma }_2^2)$

Here ${\sigma }_1^2$ ("sigma-one-squared") is the variance = spread-squared of the first noise; likewise ${\sigma }_2^2$. The rule says the variances add — not the spreads themselves.

WHY it is Pythagoras. Independent noises point in "unrelated directions". Two unrelated directions are geometrically like perpendicular vectors. The length of the sum of two perpendicular vectors is $\sqrt{{\sigma }_1^2+{\sigma }_2^2}$ — exactly the hypotenuse. This is the single tool that lets us merge two rounds of static into one.

PICTURE. Two perpendicular grey noise-arrows; the diagonal (their sum) has length equal to the square-root of the sum of their squared lengths.

---

Step 3 — Two smudges in a row, then merge them

WHAT. Substitute the step-$(t-1)$ rule into the step-$t$ rule (unroll one layer): $x_t=\sqrt{{\alpha }_t}\left(\underset{=x_{t-1}}{\underbrace{\sqrt{{\alpha }_{t-1}}{x}_{t-2}+\sqrt{1-{\alpha }_{t-1}}{ϵ}_{t-2}}}\right)+\sqrt{1-{\alpha }_t}{ϵ}_{t-1}$

Distribute the outer $\sqrt{{\alpha }_t}$: $x_t=\underset{\text{signal, twice shrunk}}{\underbrace{\sqrt{{\alpha }_t{\alpha }_{t-1}}{x}_{t-2}}}+\underset{\text{two separate noises}}{\underbrace{\sqrt{{\alpha }_t(1-{\alpha }_{t-1})}{ϵ}_{t-2}+\sqrt{1-{\alpha }_t}{ϵ}_{t-1}}}$

WHY merge. Two noise terms are ugly; we want one. Use Step 2. The first noise has variance ${\alpha }_t(1-{\alpha }_{t-1})$ (the  ​ squared), the second has variance $(1-{\alpha }_t)$. Add the variances: ${\alpha }_t(1-{\alpha }_{t-1})+(1-{\alpha }_t)={\alpha }_t-{\alpha }_t{\alpha }_{t-1}+1-{\alpha }_t=1-{\alpha }_t{\alpha }_{t-1}$

So the two noises collapse into a single fresh noise $ϵ$ of variance $1-{\alpha }_t{\alpha }_{t-1}$: $x_t=\sqrt{{\alpha }_t{\alpha }_{t-1}}{x}_{t-2}+\sqrt{1-{\alpha }_t{\alpha }_{t-1}}ϵ$

PICTURE. The two grey noise-arrows from the two steps fuse into one diagonal arrow; the surviving-signal multiplier becomes the product ${\alpha }_t{\alpha }_{t-1}$ (each step shrinks, and shrinks stack multiplicatively).

---

Step 4 — Keep unrolling → the product ${\bar{\alpha }}_t$ is born

WHAT. Nothing new happens if we unroll a third, fourth, ... step: each time the surviving-signal factor picks up one more $\alpha$, and all the noises keep merging into one. After going all the way back to $x_0$: $x_t=\sqrt{{\alpha }_t{\alpha }_{t-1}\cdots {\alpha }_1}{x}_0+\sqrt{1-{\alpha }_t{\alpha }_{t-1}\cdots {\alpha }_1}ϵ$

WHY a name. That long product appears twice, so we name it: ${\bar{\alpha }}_t:={\prod }_{s=1}^t{\alpha }_s={\alpha }_1{\alpha }_2\cdots {\alpha }_t$ The bar means "cumulative". ${\bar{\alpha }}_t$ is the fraction of the original signal still alive after $t$ steps. Since each ${\alpha }_s<1$, multiplying them makes ${\bar{\alpha }}_t$ shrink toward 0 — the picture fades.

PICTURE. A chain of shrinking blue signal-arrows (each multiplied by another ${\alpha }_s$) versus a growing grey noise-cloud; where the two cross is the "half signal, half noise" moment.

---

Step 5 — The one-line direct-jump formula

WHAT. Substituting the name gives the central result of the whole topic: $\boxed{x_t=\underset{\text{signal weight}}{\underbrace{\sqrt{{\bar{\alpha }}_t}}}{x}_0+\underset{\text{noise weight}}{\underbrace{\sqrt{1-{\bar{\alpha }}_t}}}ϵ}ϵ\sim \mathcal{N}(0,I)$

WHY this is a big deal. Naively, to reach step $t=800$ you would sample $800$ times in sequence. This formula lets you teleport to any step in a single draw of $ϵ$ — essential for training, where you pick a random $t$ every batch. The signal weight $\sqrt{{\bar{\alpha }}_t}$ and noise weight $\sqrt{1-{\bar{\alpha }}_t}$ always satisfy $(\sqrt{{\bar{\alpha }}_t}{)}^2+(\sqrt{1-{\bar{\alpha }}_t}{)}^2=1$: the picture rotates from "all signal" to "all noise" along a unit circle.

PICTURE. A single unit vector making angle ${\varphi }_t$ with the signal axis: its horizontal shadow is $\sqrt{{\bar{\alpha }}_t}$ (signal), its vertical shadow is $\sqrt{1-{\bar{\alpha }}_t}$ (noise). As $t$ grows, the vector rotates from horizontal (clean) to vertical (pure noise).

---

Step 6 — Edge case A: the start ($t\to 0$), and why ${\beta }_1=0$ is forbidden

WHAT. At the very start, ${\bar{\alpha }}_0=1$ (empty product = 1), so $x_0=x_0$ — no noise. Tempting to set ${\beta }_1=0$ so the first step also does nothing.

WHY it breaks. Look at the SNR: $\text{SNR}(0)=\frac{{\bar{\alpha }}_0}{1-{\bar{\alpha }}_0}=\frac{1}{1-1}=\frac{1}{0}=\infty$ Dividing by zero is undefined; the log-SNR blows up to $+\infty$, and the training loss (which divides by the noise variance) explodes. The fix is a tiny ${\beta }_1\approx 10^{-4}$, or clipping ${\bar{\alpha }}_0$ to $1-\epsilon$, so the noise shadow is small-but-nonzero and every quantity stays finite.

PICTURE. The rotating vector pinned exactly horizontal: its vertical (noise) shadow is zero, so the ratio's denominator collapses — we mark the singularity in red, then show the "nudged-off-horizontal" safe version.

---

Step 7 — Edge case B: the end ($t\to T$), and why we insist ${\bar{\alpha }}_T\approx 0$

WHAT. At the last step we want $x_T$ to be pure static, indistinguishable from $\mathcal{N}(0,I)$, so that sampling can start from pure noise. That requires the signal weight $\sqrt{{\bar{\alpha }}_T}\approx 0$, i.e. ${\bar{\alpha }}_T\approx 0$ (aim for $<10^{-2}$).

WHY it must not overshoot. If a linear schedule drives ${\bar{\alpha }}_t$ to near-zero too early (say by $t=500$ out of $1000$), the last 500 steps are all "pure noise → pure noise" — wasted work. This is precisely why the cosine schedule wins: it keeps the vector rotating evenly and only reaches vertical right at $t=T$.

PICTURE. Two rotation paths for ${\bar{\alpha }}_t$ across $t$: the linear one plummets and flattens on the noise floor early (wasted region shaded), the cosine one glides down evenly and lands on the floor exactly at $T$.

---

The one-picture summary

Everything above compresses into a single diagram: a unit vector sweeping from the signal axis to the noise axis as $t$ runs $0\to T$. Its horizontal shadow is $\sqrt{{\bar{\alpha }}_t}$, its vertical shadow is $\sqrt{1-{\bar{\alpha }}_t}$, and ${\bar{\alpha }}_t=\prod {\alpha }_s$ is the running product that drives the sweep. Good schedules make the angle advance evenly (steady difficulty); the two ends are guarded so we never divide by zero (start) and always reach pure noise (end).

Recall Feynman retelling — say it back in plain words

We wanted to add static to a picture over and over, a thousand tiny times. Adding it one step at a time is slow, so we asked: can we skip straight to any step? Yes — because independent static combines like the sides of a right triangle (variances add), every step's noise fuses into one scoop of static, and every step's shrinking of the signal multiplies into one number ${\bar{\alpha }}_t$. So the noisy picture is just $\sqrt{{\bar{\alpha }}_t}$ of the original plus $\sqrt{1-{\bar{\alpha }}_t}$ of fresh static — a single vector rotating from "all picture" to "all snow". We must not start exactly on the picture axis (dividing by zero static breaks the math) and we must finish exactly on the snow axis (so sampling can begin from pure noise). Choosing how fast the vector rotates is the whole art of noise scheduling.

Where this leads

• 4.5.11-Forward-and-reverse-diffusion-process — the single-step rule in Step 1 is the forward process $q(x_t|x_{t-1})$.
• 4.5.13-Denoising-objective — the loss is built on predicting the $ϵ$ in the Step 5 formula.
• 4.5.15-Score-matching — the SNR in Step 5 sets the magnitude of the score ${\nabla }_{x_t}\log p(x_t)$.
• 4.5.18-Fast-sampling — because Step 5 lets us jump to any $t$, we can skip steps at sampling time (DDIM).

Recall Quick self-test

What does ${\bar{\alpha }}_t$ represent physically? ::: The fraction of the original signal's variance still surviving after $t$ noising steps; it is the product ${\prod }_{s=1}^t{\alpha }_s$. Why square-roots in $x_t=\sqrt{{\alpha }_t}{x}_{t-1}+\sqrt{1-{\alpha }_t}ϵ$? ::: Because variance scales with the square of a multiplier, so  ​ makes the signal and noise variances sum to exactly 1. Why can't ${\beta }_1=0$? ::: Then ${\bar{\alpha }}_0=1$, so $\text{SNR}=1/0=\infty$ and the loss divides by zero — undefined. What do we require of ${\bar{\alpha }}_T$? ::: It must be $\approx 0$ (below $10^{-2}$) so $x_T$ is indistinguishable from the prior $\mathcal{N}(0,I)$.