Foundations — Prompt engineering best practices

Before you can read the parent note fluently, you need to see what each symbol and word means. This page builds every one of them from absolute zero. We go slowly and in order — each idea leans only on the ones above it.

0. What is "text" to a machine? — the token

The picture: imagine a long train, and each carriage holds one token. The model reads the carriages left to right.

Why the topic needs it: the whole game of prompting is choosing which carriages to put at the front of the train. When the parent note says "next-token predictor," token is that carriage.

1. The symbol $x_t$ — one token at a time

So $x_1$ is the first token, $x_2$ the second, and so on. The little $t$ underneath is just a label saying "which one."

The picture: put a number under each carriage in the train above. $x_3$ = "the token in carriage 3."

Why the topic needs it: the model generates text one carriage at a time, left to right. To talk about "the next one," we must be able to point at "position $t$."

2. The symbol $x_{<t}$ — "everything before now"

The tiny <t is literally saying "positions less than $t$."

The picture: stand at carriage $t$ and look backwards. Everything behind you is $x_{<t}$.

Why the topic needs it: the model decides the next token by looking only at what came before it. That backward-looking pile of text is exactly $x_{<t}$.

3. The bar $\mid$ — "given that…"

The picture: a spinner whose slices resize depending on the context you feed it.

Why the topic needs it: when the parent writes $P(\text{answer}\mid \text{prompt})$, it is saying "the probability of an answer, given your prompt." Rewriting the prompt reshapes the spinner.

4. The distribution — a spinner over all possible next tokens

The picture (see figure above): the spinner has one slice per possible next word. A big slice = "very likely next." The model "samples" by spinning and reading where it lands.

Why the topic needs it: "steering the distribution" (parent's phrase) literally means "reshaping this spinner" toward the slice you want.

5. The product symbol $\prod$ — building a sentence token by token

So the parent formula $P(\text{answer}\mid \text{prompt})={\prod }_{t}P(x_t\mid x_{<t},\text{prompt})$ reads in plain words: "the chance of the whole answer is the chance of token 1 (given the prompt), times the chance of token 2 (given the prompt and token 1), times the chance of token 3, … all the way to the end."

The picture: a chain of links. Each link is one token's probability; multiply them all to get the strength of the whole chain.

Why the topic needs it: it shows why generation is left-to-right and step-by-step, which is the deep reason Chain-of-Thought works — more tokens means more links to spread hard computation across.

6. The context window — how far back the model can see

The picture: a fixed-width viewing frame sliding over the token train. Only carriages inside the frame count as $x_{<t}$.

Why the topic needs it: the prompt lives inside this window. It explains the "lost in the middle" mistake — the model attends less to carriages sitting in the middle of a very long frame, so key instructions go at the start and end.

7. $k$ — the number of examples (shots)

The picture: $k$ demonstration cards laid on the table before you ask the real question.

Why the topic needs it: few-shot prompting and in-context learning are defined by $k$. The examples reshape the spinner without ever changing the model's internal numbers ("no weight update").

8. Putting it together — the prerequisite map

Each foundation feeds the next, and together they feed the topic:

Related deeper pages once these click: Temperature and sampling (how the spin actually picks a slice), Retrieval-Augmented Generation (RAG) (stuffing facts into the window), Prompt injection & LLM security (attacks through the same input channel), Hallucination in LLMs (confident wrong slices), and Alignment & RLHF (how the spinner got shaped in training).

Equipment checklist

Test yourself — cover the right side and answer aloud.