Exercises — T5 and text-to-text framework

(a) output string — labels are printed as text, not chosen by a label head. (b) neither — T5 has no regression head; this is exactly what it removes. (c) input string — the prefix "summarize:" plus the article is the encoder input. (d) output string — emitted as the characters "3.8", rounded to the nearest $0.2$.

T5 is encoder–decoder. The two it is not: encoder-only = BERT and masked language modeling; decoder-only = GPT and decoder-only models. The generic backbone is the Transformer encoder-decoder architecture.

Replace each contiguous dropped span with one sentinel, then list sentinel-then-original-text in the target, closing with the next unused sentinel.

• Input: the <X> sat <Y> mat
• Target: <X> cat <Y> on the <Z>

<X> stands for the single word cat; <Y> stands for the whole two-word span on the; the trailing <Z> marks the end of the target (nothing follows it).

Plug straight into the formula — sum of $-\ln p_t$: $\mathcal{L}=-(\ln 0.5+\ln 0.8+\ln 0.25+\ln 1.0).$ Numerically: $-\ln 0.5=0.6931$, $-\ln 0.8=0.2231$, $-\ln 0.25=1.3863$, $-\ln 1.0=0$. $\mathcal{L}=0.6931+0.2231+1.3863+0=2.3026.$ Per-token average $=\mathcal{L}/4=0.5757$. Notice the </s> token contributed $0$: the model was perfectly sure the sentence ended, so it added no loss.

Round $3.83$ to the nearest multiple of $0.2$. The multiples near it are $3.8$ and $4.0$; $3.83$ is $0.03$ from $3.8$ and $0.17$ from $4.0$, so it snaps to $3.8$. Target string $=$ "3.8". (Rounding to $0.2$ keeps the set of possible targets finite — $0.0,0.2,\dots ,5.0$, i.e. 26 possible strings — so it stays a clean language-modeling problem.)

• BERT (BERT and masked language modeling): the output at a masked position is a single softmax over the vocabulary — one classification per masked slot, independent of the others. Its output length is fixed = number of masks.
• T5: the output is a generated sequence <X> span <Y> span …. The decoder emits tokens autoregressively, so a span can decode to any number of tokens. Therefore T5 can output variable-length answers per region; BERT cannot (one slot ⇒ one token). This is exactly why T5's loss sums over a target sequence while BERT's sums over fixed positions.

B counts as wrong (accuracy $0$ on this example). At inference T5 just reads the produced string and checks whether it equals a valid label; "positve" is not a label, so it is scored incorrect — same as an outright wrong answer. This is the honest behavior: the model was asked to produce the label string and did not, so treating a malformed string as "close enough" would inflate accuracy. In practice such off-vocabulary outputs are rare after fine-tuning because the label strings dominate the target distribution.

• (i) Prefix: "spam classify: ".
• (ii) Input: "spam classify: You WON a free prize, click here!!!".
• (iii) Targets: "spam" or "ham" (two label strings).
• (iv) The model still receives a string and emits a string; the same encoder reads it, the same decoder generates, the same cross-entropy over target tokens trains it. Only the data changes (different prefix + label vocabulary), never the architecture or loss. This is the Multi-task learning with prefixes payoff — see also Transfer learning and fine-tuning.

Mini-batch:

• "translate English to German: Good morning." → "Guten Morgen."
• "spam classify: Claim your reward now!" → "spam"

The model knows the task purely from the prefix at the start of the input — there is no task-ID input, no per-task head. Because both examples are string → string scored by the same loss, they can share a gradient step. This is exactly Prompting and instruction tuning in seed form: the instruction lives inside the text.

Because $-\ln (\cdot )$ is a strictly decreasing function of its argument on $(0,1]$, whatever value of the parameters maximizes $p(y\mid x)$ simultaneously minimizes $-\ln p(y\mid x)$. No maximum of $p$ can move without $-\ln p$ moving in the opposite direction, so the two problems share the same optimizer. The log also turns the product ${\prod }_{t}p(y_t\mid \cdot )$ into a sum ${\sum }_t-\ln p(y_t\mid \cdot )$, which is what makes the multi-token loss additive and numerically stable. Numerically $-\ln 0.4=0.91629\dots$ ✓.

Model P: ${\mathcal{L}}_P=2.3026$ (from 2.2), per-token $0.5757$, so ${\text{PPL}}_P=\exp (0.5757)=1.778$. Model Q: ${\mathcal{L}}_Q=-(\ln 0.9+\ln 0.9+\ln 0.9+\ln 1.0)=3(0.10536)=0.31608$, per-token $0.07902$, ${\text{PPL}}_Q=\exp (0.07902)=1.0822$. Q's loss is lower by $2.3026-0.3161=1.9865$. Perplexity reads as "effective number of equally-likely choices per token": Q is near $1.08$ (almost certain each step), P is $1.78$ (roughly like flip-a-slightly-loaded-coin uncertainty). Lower perplexity = crisper predictions.

Refute (partially). Cross-entropy scores each token against the target token. Target "3.8" tokenizes to (roughly) 3, ., 8. A prediction of 4.0 gets the first digit and the last digit wrong; 4.8 gets the first digit wrong but the last digit 8 right. So token-wise they are not equally wrong — 4.8 can incur less cross-entropy because it matches one more target token. However, cross-entropy has no notion of numeric distance: it does not know that $4.0$ is closer to $3.8$ than $4.8$ is. So the metric rewards character overlap, not numeric proximity — a genuine limitation of text-as-number regression, and the reason rounding to a small fixed grid ($0.2$ steps) matters.