Worked examples — Teacher forcing

This page is a drill sheet. The parent note told you what teacher forcing is. Here we hunt down every distinct situation it can produce and work each one by hand, so no exam question or real project surprises you.

Before we start, one reminder of the only numbers we ever compute here.

Look at the red curve: it is flat and low on the right (confident-correct is cheap) and explodes on the left (a tiny probability on the truth is catastrophic). Every "cascade" story below is just this curve being visited at a bad $p$.

The teacher-forcing ratio $\epsilon \in [0,1]$ is the probability that, on any given step, we feed the model the ground-truth previous token instead of its own prediction. $\epsilon =1$ = full training wheels; $\epsilon =0$ = no wheels.

---

The scenario matrix

Every question about teacher forcing lands in one of these cells. The examples below are labelled with the cell they cover.

#	Cell (the scenario class)	What makes it distinct	Example
A	$\epsilon =1$, model always correct	best case, clean gradients	Ex 1
B	$\epsilon =0$, error at one step	cascade / compounding	Ex 2
C	Same weights, TF loss vs inference loss	exposure-bias gap	Ex 3
D	$0<\epsilon <1$ (scheduled sampling)	expected loss over coin flips	Ex 4
E	Degenerate: $T=1$ sequence	no history exists at all	Ex 5
F	Limiting: $p\to 0$ on truth	loss $\to \infty$, why clip	Ex 6
G	Real-world word problem (translation)	encoder context + free-run	Ex 7
H	Exam twist: decay schedule algebra	find epoch where $\epsilon$ hits floor	Ex 8

---

Worked examples

Forecast: guess — will this total be closer to $0.5$, $0.9$, or $3$?

1. Write the per-step loss. Why this step? Loss is defined only on the probability of the true target, and each step is independent because with $\epsilon =1$ every input is ground truth. ${\ell }_t=-\log p_t$
2. Plug numbers. Why this step? Direct substitution — no history to worry about. $-\log 0.7=0.357,-\log 0.8=0.223,-\log 0.9=0.105,$ $-\log 0.85=0.163,-\log 0.95=0.051$
3. Sum. Why this step? Total sequence loss is the sum over timesteps (from the chain-rule factorization: log of a product is a sum of logs). ${\mathcal{L}}_{TF}=0.357+0.223+0.105+0.163+0.051=0.899$

Verify: every $p>0$, so every term is positive and finite ✓. Average per-token loss $=0.899/5=0.18$, and $e^{-0.18}\approx 0.84$ — a sensible "average confidence" between $0.7$ and $0.95$ ✓.

---

Forecast: bigger or smaller than Example 1's $0.899$? By how much?

1. Compute clean early steps. Why? The first steps still happen to be right, so their loss matches the TF numbers' shape. $-\log 0.7=0.357,-\log 0.8=0.223,-\log 0.6=0.511$
2. The mistake step $t=4$. Why? Because at $t=3$ the model already fed itself a shaky input, its confidence on truth L collapsed to $0.35$. $-\log 0.35=1.050$
3. The cascade step $t=5$. Why? Having emitted a wrong token, the model is now in a state it never trained on — probability on truth O crashes to $0.2$. $-\log 0.2=1.609$
4. Sum. Why? Same additive rule; the point is where the mass went. ${\mathcal{L}}_{FR}=0.357+0.223+0.511+1.050+1.609=3.750$

Verify: $3.750/0.899\approx 4.2$ — over $4\times$ worse, driven entirely by the last two terms sitting on the steep left arm of the loss curve in figure s01 ✓.

The bar chart makes the cascade visible: the first three bars barely differ between TF and FR; the last two tower under free running.

---

Forecast: is the gap additive (a subtraction) or multiplicative (a ratio) — and which is the "honest" number?

1. Define the gap. Why? Exposure bias is literally the difference between the distribution the model was trained on (ground-truth history) and the one it faces at test time (own history). $\Delta ={\mathcal{L}}_{\text{inference}}-{\mathcal{L}}_{TF}$
2. Plug in. Why? Both numbers are the same network, only the input source differs. $\Delta =3.750-0.899=2.851$
3. Per-token view. Why? Absolute gaps grow with $T$; per-token normalizes so you can compare sequences of different lengths. ${\Delta }_{\text{per token}}=2.851/5=0.570$

Verify: $\Delta >0$ (inference must be at least as hard as training) ✓. The whole gap ($2.851$) equals the extra loss on steps 3–5 only: $(0.511-0.105)+(1.050-0.163)+(1.609-0.051)=0.406+0.887+1.558=2.851$ ✓ — confirming the gap lives entirely where histories diverged.

---

Forecast: halfway between $0.9$ and $1.6$? Guess the exact value.

1. Write the expectation. Why? A random coin (the $\epsilon$-flip) means the realized loss is a random variable; we report its mean. $\mathbb{E}[\ell ]=\epsilon \cdot {\ell }_{\text{truth}}+(1-\epsilon )\cdot {\ell }_{\text{pred}}$
2. Substitute $\epsilon =0.5$. Why? This is the curriculum "half-and-half" regime — the model trains on both clean and self-generated histories. $\mathbb{E}[\ell ]=0.5(0.9)+0.5(1.6)=0.45+0.80=1.25$

Verify: $1.25$ lies strictly between $0.9$ and $1.6$ ✓. Sanity limits: at $\epsilon =1$ formula gives $0.9$ (pure TF), at $\epsilon =0$ gives $1.6$ (pure free-run) ✓.

---

Forecast: true or false — "$\epsilon$ matters here."

1. List the history. Why? Teacher forcing only ever affects the input fed as previous token; here $t=1$ has only the fixed <START>, which is ground truth by definition. $y_{1:0}=()\text{(empty)}$
2. Compute the loss. Why? One term, no cascade possible. $\ell =-\log 0.6=0.511$
3. Conclude. Why? Since no step ever consumes a predicted previous token, TF and free-run are identical.

Verify: setting $\epsilon =1$ or $\epsilon =0$ both give $0.511$ ✓. The exposure-bias gap $\Delta =0$ for $T=1$ ✓ — degenerate case behaves as expected (no history to be biased about).

---

Forecast: roughly $1$, roughly $9$, or "infinite"?

1. Evaluate. Why? Just read off the loss curve at a tiny $p$ — the steep left arm. $\ell =-\log (10^{-4})=4\log 10=9.210$
2. Take the limit. Why? To see the failure mode of the exact formula. ${\lim }_{p\to 0^{+}}(-\log p)=+\infty$
3. The fix. Why? An infinite loss gives infinite gradient and NaNs, so implementations clamp $p\ge p_{\min }$ (e.g. $10^{-9}$) or add a tiny $\eta$: $-\log (p+\eta )$.

Verify: $-\log (10^{-4})=9.210$ and $e^{-9.210}=10^{-4}$ ✓ round-trips. Clamped at $p_{\min }=10^{-9}$: $-\log (10^{-9})=20.72$, finite ✓.

---

Forecast: will the training loss beat Example 1's $0.899$?

1. Training loss (TF). Why? Encoder gives a fixed context $c$; each decoder step still gets ground truth as previous token. $\mathcal{L}=-\log 0.9-\log 0.85-\log 0.95=0.105+0.163+0.051=0.319$
2. Inference divergence. Why? At test time no truth exists; the decoder feeds its own argmax. If $t=1$ argmax were You, then at $t=2$ the model conditions on You, a history it never trained on → possible "You is" nonsense.
3. Takeaway. Why? This is exactly the train/test mismatch of Cell C, now in a real MT setting.

Verify: $0.319<0.899$ ✓ (shorter, more confident sequence). $e^{-0.319/3}=e^{-0.106}=0.899$, matching the geometric-mean confidence $\sqrt[3]{0.9\cdot 0.85\cdot 0.95}=0.899$ ✓.

---

Forecast: epoch $25$, $50$, or $100$?

1. Solve the linear part $=0.5$. Why? The $\max$ floor kicks in exactly when the decaying line crosses $0.5$. $1.0-0.01k=0.5\Rightarrow 0.01k=0.5\Rightarrow k=50$
2. Evaluate at $k=30$. Why? $30<50$, so we are still on the decaying line, not the floor. ${\epsilon }_{30}=1.0-0.01(30)=0.70$

Verify: at $k=50$, $1-0.01(50)=0.5$ so $\max (0.5,0.5)=0.5$ — floor just reached ✓. At $k=60$, line gives $0.4$ but $\max (0.5,0.4)=0.5$, so it stays floored ✓. ${\epsilon }_{30}=0.70\in [0.5,1]$ ✓.

---

Recall Quick self-test

With $\epsilon =1$ and probs $0.7,0.8,0.9,0.85,0.95$ the total loss is? ::: $0.899$ The exposure-bias gap in Ex 3 (inference minus TF) is? ::: $2.851$ Expected step loss at $\epsilon =0.5$ mixing $0.9$ and $1.6$? ::: $1.25$ Epoch where ${\epsilon }_k=1-0.01k$ first hits the $0.5$ floor? ::: $50$ For a $T=1$ sequence, does $\epsilon$ change the loss? ::: No — there is no predicted history to feed.