Teacher forcing: Question bank — AI-ML deep dive

Before we start, five symbols you must own, because every trap leans on them:

Definition The five load-bearing pieces of notation

Autoregressive :: the model predicts token $t$ using the tokens before it as input, so each output feeds the next input. Picture a chain where every link is forged from the previous link.
Teacher forcing :: during training you feed the model the ground-truth previous token $y_{t - 1}$ instead of its own guess $\overset{y}{^}_{t - 1}$ . Training wheels.
Exposure bias :: the mismatch between training (always fed correct history) and inference (fed its own, possibly wrong, history). The model is never exposed to its own mistakes while learning.
== $θ$ (theta)== :: the model's parameters — every weight and bias inside the network, bundled into one symbol. Writing $p_{θ} (\cdot)$ means "the probability the model gives, using its current knobs set to $θ$ ." Training = searching for the $θ$ that makes the true sequences most likely.
== $ϵ$ (epsilon), the teacher-forcing ratio== :: a number in $[0, 1]$ that is the probability of using ground truth at each step. $ϵ = 1$ means always use ground truth (pure teacher forcing); $ϵ = 0$ means always use the model's own prediction (pure free-running); in between is scheduled sampling. Because it is a probability it can never exceed $1$ .

The next two figures make the visual claims of this page concrete: how a discrete $ar g max$ blocks gradient flow, and how one wrong token cascades under free-running while teacher forcing resets each step.

True or false — justify

Teacher forcing is a technique used at inference time to speed up generation.

False. It is a training-only technique. At inference there is no ground truth to feed, so the model must consume its own predictions — that is precisely the situation teacher forcing hides during training.

Teacher forcing changes the loss function we optimise.

False. The loss is still the same per-token cross-entropy

\sum_{t} - lo g p_{θ} (y_{t} ∣ \cdot, x)

(recall

θ

is the model's parameters); what teacher forcing changes is what history conditions each term — ground truth instead of predictions.

With teacher forcing, gradients still have to flow backward through the discrete

ar g max

of every previous step.

False. Because the previous input is fixed ground-truth data (not a sampled/argmax'd prediction), there is no non-differentiable

ar g max

in the input path. That is exactly why teacher-forced training is clean and fast.

A model trained with pure teacher forcing (

ϵ = 1

) will always generate perfectly at inference.

False. It was never shown its own errors, so a single wrong token at test time puts it in an unseen state and the mistake can cascade — this is exposure bias in action.

Setting

ϵ = 0

(pure free-running) removes exposure bias entirely.

True in spirit but misleading. Training conditions now match inference, but the training becomes slow and unstable because early random errors poison every later step's context, so pure

ϵ = 0

is rarely usable from scratch.

Teacher forcing and seq2seq are the same thing.

False. Seq2seq is an architecture (encoder–decoder); teacher forcing is a training strategy you may apply to its decoder. You can run seq2seq with or without teacher forcing.

The chain-rule factorisation

p (y ∣ x) = \prod_{t} p (y_{t} ∣ y_{1 : t - 1}, x)

is only valid when teacher forcing is used.

False. That factorisation is a pure identity of probability (true for any joint distribution — see the staircase figure). Teacher forcing is merely one choice of what to plug in for

y_{1 : t - 1}

during training.

Scheduled sampling with a decaying

ϵ

is a form of curriculum learning.

True. It starts easy (full ground-truth support at

ϵ = 1

) and gradually raises difficulty (more self-generated history as

ϵ

falls), which is the defining shape of curriculum learning.

Spot the error

"At inference we should still feed the ground-truth previous token so the outputs stay accurate." — what is wrong?

There is no ground truth at inference; that is the whole point of generation. You only have your own previous outputs, so the ground-truth token is unavailable by definition.

"Free-running training is fine because cross-entropy is differentiable." — where's the flaw?

Cross-entropy is differentiable, but choosing the next input via

ar g max

over the previous output is not differentiable (see the flat-staircase figure), so gradients cannot flow cleanly through the input chain the way they can under teacher forcing.

"Exposure bias means the model has seen too much training data." — correct the statement.

No — exposure bias is the opposite kind of gap: at training the model is only ever exposed to correct histories, so at test time it is under-exposed to its own erroneous states. See 5.3.8-Exposure-bias.

"To eliminate exposure bias, just train longer with

ϵ = 1

." — why does this fail?

More epochs at

ϵ = 1

only sharpen behaviour on ground-truth histories; it never introduces the model's own mistakes, so the train–test distribution gap is untouched no matter how long you train.

"Teacher forcing ratio

ϵ = 1.2

gives extra-strong supervision." — spot the mistake.

ϵ

is a probability (the chance of using ground truth) and must lie in

[0, 1]

. A value above 1 is meaningless;

ϵ = 1

is already the maximum (always use ground truth).

"Since teacher forcing gave lower training loss than free-running in the worked example, teacher forcing is the better model." — what's the confusion?

Lower training loss under teacher forcing does not mean better generation; it can hide exposure bias. The two losses measure different conditioning distributions, so comparing them as "which model is better" is a category error.

"In machine translation the encoder also uses teacher forcing." — is that right?

No. Teacher forcing applies to the autoregressive decoder, which generates one token at a time. The encoder reads the whole source at once and has no such step-by-step self-conditioning.

Why questions

Why do we take the logarithm of

p (y ∣ x)

before optimising?

The log turns the product

\prod_{t} p (y_{t} ∣ \cdot)

into a sum

\sum_{t} lo g p (y_{t} ∣ \cdot)

, which is numerically stable and easy to differentiate term-by-term; it also equals the negative cross-entropy loss we already use.

Why does teacher forcing make training faster to converge, not just more stable?

Every timestep receives the correct context immediately, so the model learns each conditional

p_{θ} (y_{t} ∣ y_{1 : t - 1})

independently and cleanly rather than waiting to first stumble onto good early tokens by itself.

Why does a single wrong token at inference cause a cascade rather than a one-off error?

The wrong token becomes the input to the next step, pushing the model into a state it never trained on; its next prediction is therefore unreliable, and that new error feeds forward again — the autoregressive loop amplifies mistakes (see the cascade figure).

Why is scheduled sampling a compromise rather than a strict improvement?

It reintroduces the model's own (sometimes wrong) tokens during training so the loss target may be inconsistent with the sampled input, which can bias gradients; it trades some training stability to shrink the exposure-bias gap.

Why can't we simply define the training loss to include our own predictions to match inference exactly?

Doing so requires differentiating through the discrete sampling/argmax that selects each next input, which is non-differentiable, so standard backprop cannot compute honest gradients through that mismatch.

Why do autoregressive models suffer exposure bias while a non-autoregressive model might not?

Exposure bias arises because each prediction feeds the next input; a model that predicts all tokens in parallel with no self-conditioning has no error-feedback loop to accumulate mistakes.

Edge cases

What happens with a sequence of length

T = 1

(a single token)?

There is no previous token to feed, so teacher forcing and free-running are identical — no exposure bias can arise because there is no self-conditioning chain to build.

ϵ

is drawn per-timestep rather than per-sequence, what changes at the boundaries?

Within one sequence some steps use ground truth and some use predictions, so the history a later step sees is a mix; this samples train-time histories closer to the ragged ones seen at inference, but makes each step's context depend on earlier coin flips.

For the very first decoder step (

t = 1

), what is the "previous token", and does teacher forcing matter there?

The input is a fixed <START> token, identical under both strategies. Teacher forcing has no effect at

t = 1

because there is no earlier predicted token to replace — the distinction only appears from

t = 2

onward.

What is the degenerate behaviour when the model is already perfect (predicts every ground-truth token with probability 1)?

Teacher forcing and free-running coincide, since

ar g max

of the prediction equals the ground truth at every step; exposure bias vanishes because the self-generated history is exactly the correct history.

If the training data itself contains a wrong/noisy target token, what does teacher forcing do with it?

It faithfully feeds that noisy token as the "correct" history at the next step, so teacher forcing can propagate label noise forward — it trusts the data completely and has no mechanism to question a bad ground-truth token.

Recall Fast self-test

Teacher forcing feeds the model ______ at each step during training. ::: the ground-truth previous token $y_{t - 1}$ The gap between training on truth and inferring on predictions is called ______. ::: exposure bias In $p_{θ} (\cdot)$ , the symbol $θ$ stands for ______. ::: all the model's parameters (weights and biases) The teacher-forcing ratio $ϵ$ must lie in the interval ______. ::: $[0, 1]$ , because it is a probability Mixing ground truth and predictions with a probability $ϵ$ is called ______. ::: scheduled sampling The non-differentiable operation that blocks clean gradients in free-running is ______. ::: the $ar g max$ (discrete selection) of the previous output