Foundations — Supervised fine-tuning (SFT)

Before you can read the parent note SFT, you need to own every symbol it throws at you. This page builds each one from nothing, in an order where each brick rests on the one before it. Nothing here assumes you've seen probability notation, logs, or the $\sum$/$\prod$ signs.

0. The picture everything sits on: a token stream

Everything in language modeling starts with one idea: text is chopped into little pieces called tokens (roughly words or word-fragments), and the model reads them left-to-right.

We will call the pieces before some position "the past" and the piece we're about to guess "the next token". That splitting is the heart of everything.

1. The vocabulary $V$

The little symbol $\in$ just means "is a member of" — read $v\in V$ as "v is one of the tokens on the shelf".

2. Probability, and why $p_v$ appears everywhere

The model never says "the next token is definitely X". Instead it hands you a confidence for every token on the shelf.

The notation $p_{y_t}$ (you'll meet $y_t$ in a second) just means: the bar height the model gave to the token that was actually correct.

3. Naming the sequences: $x$, $y$, $y_t$, $y_{<t}$

Now we name the two things every SFT example contains: the question and the ideal answer.

The subscript $t$ is just a position counter, like "the 3rd bead". So $y_3$ = "the 3rd answer token".

4. The conditional: ${\pi }_{\theta }(y_t\mid x,y_{<t})$

This is the single most important expression in the parent note. Let's disassemble it left to right.

• ${\pi }_{\theta }$ — the model itself. $\pi$ (Greek "pi") is just the name we give the machine; $\theta$ (Greek "theta") is the pile of numbers inside it (its weights) that training adjusts.
• The bar $\mid$ means "given" — read it as "assuming we already know...".
• So ${\pi }_{\theta }(y_t\mid x,y_{<t})$ reads:

"The probability the model gives to the answer-token $y_t$, given the prompt $x$ and all earlier answer tokens $y_{<t}$."

5. The product sign $\prod$ — probability of a whole answer

To score a whole answer, we combine its per-token probabilities. The rule (the "chain rule of probability") says: multiply them.

6. The logarithm — turning multiply into add

Multiplying many numbers below $1$ gives absurdly tiny values (a 100-token answer might have probability like $0.0000\dots$ with 40 zeros). Computers choke on that. The logarithm rescues us.

One more fact we'll use: for any probability $p$ between $0$ and $1$, $\log p$ is negative or zero (e.g. $\log 0.5=-0.693$, $\log 1=0$). Confident-and-correct → close to $0$; unsure → very negative.

7. The minus sign — from "score to grow" to "loss to shrink"

Optimizers are built to push numbers down. But we want to push probability up. Fix: flip the sign.

This "$-\log p_{y_t}$" is exactly the cross-entropy the parent note keeps referring to. It's the same number pretraining uses; SFT just feeds it curated data.

8. The indicator $1[\cdot ]$ and the mask $m_t$

Two look-alike symbols in the parent note that are easy to confuse.

9. The averaging symbols: $\mathbb{E}$, $\mathcal{D}$, and the $\frac{1}{\sum m_t}$

How these foundations feed the topic

Equipment checklist