Zero-shot and few-shot prompting: Visual walkthrough

We are unpacking the central formula from the parent note:

$answer = ar g max_{y} P_{θ} (y ∣ demos, query)$

Two words in that line need naming before we go further, because everything hinges on them:

Step 1 — What is a "next-token predictor", really?

WHAT. A large language model (LLM) is a machine that, given some text so far, outputs a guess for the very next chunk of text — one "token" at a time. A token is just a piece of a word (roughly ¾ of a word on average). Think of it as an extremely well-read autocomplete.

WHY start here. Everything about prompting is downstream of this one fact. If you don't picture what the model actually produces, phrases like "conditioning the distribution" are just noise.

PICTURE. Look at the figure. On the left is the text you have so far (call it the context). The machine reads it and, on the right, produces not a single word but a bar chart — a height for every possible next token. Tall bar = "very likely to come next"; tiny bar = "unlikely".

Figure — Zero-shot and few-shot prompting

The symbol for that list of numbers is:

$P_{θ} (x_{t} ∣ x_{1}, x_{2}, \dots, x_{t - 1})$

Term by term, right where each piece sits:

$x_{t}$ — the token at position $t$ (the one we're trying to predict now). The little $t$ is just a counter: 1st token, 2nd token, ...
$x_{1}, x_{2}, \dots, x_{t - 1}$ — every token that came before position $t$ : the context.
the bar | reads "given" — "the chance of $x_{t}$ given everything before it".
$P$ — "probability of".
$θ$ (Greek letter "theta") — the model's weights: billions of fixed numbers baked in during training. The subscript reminds us the whole bar chart is shaped by those frozen numbers.

We abbreviate $x_{1}, \dots, x_{t - 1}$ as $x_{< t}$ — "all tokens with index less than $t$ ". So the tidy form is $P_{θ} (x_{t} ∣ x_{< t})$ .

Step 2 — A whole answer is many predictions chained together

WHAT. The model writes an answer by predicting one token, gluing it to the context, predicting the next, and repeating.

WHY this tool — the chain rule. How likely is a whole sentence, not just one token? The chain rule of probability answers exactly that question: it says the probability of a sequence equals the product of each token's probability given the ones before it. We reach for it precisely because we want to go from "next token" to "whole answer".

PICTURE. The figure shows a staircase: at each step the model looks back at everything written so far (the shaded context growing left-to-right) and emits the next token. The generated tokens fold back into the context — that feedback arrow is the heart of the picture.

The one idea to carry forward: everything you type becomes part of $x_{< t}$ for every future token. Your prompt is not "outside" the machine — it is literally the left half of every prediction. Keep this chain rule in your pocket: we will need it again in Step 6, because a real answer $y$ is usually many tokens, not one.

Step 3 — "Conditioning" means: your words re-shape the bars

WHAT. Changing the context changes the bar chart. That's it. That is what "conditioning the distribution on your prompt" means — no magic, no learning.

WHY. The parent note's key claim — examples change what the model thinks is likely, not the weights — lives entirely here. We must see the bars move.

PICTURE. Two bar charts side by side, same possible tokens.

Left (empty-ish context): the model is unsure — probability is spread thin across many tokens (a flat, bumpy chart).
Right (rich context): after we add words that pin the situation down, the mass piles up on a few tokens. The chart goes peaky.

Note carefully: the weights $θ$ are identical in both charts. Only the context differs. This is why we say few-shot selects a behaviour the model already had, rather than teaching a new one.

Step 4 — Zero-shot: pinning the task with words alone

WHAT. Zero-shot means the demos part of the context is empty (from the preview above: no worked pairs, just your instruction and the query). The model must guess the task from your instruction plus whatever it saw during pretraining.

WHY it can work — and why it can wobble. For common tasks (sentiment, translation), pretraining saw millions of similar examples, so even a bare instruction peaks the chart correctly. But the instruction leaves format and label space under-specified, so the peak may land on the right idea in the wrong shape.

PICTURE. The instruction narrows the chart from "anything" down to "answers about sentiment" — but two acceptable-looking peaks remain: the clean label negative and the chatty phrase "The battery life is poor." The model might pick either.

Step 5 — Few-shot: demos collapse the chart onto one shape

WHAT. Few-shot prepends $k$ worked pairs — the demonstrations — before the real query. Formally the demo set is:

$D = {(x_{i}, y_{i})}_{i = 1}^{k}$

Term by term:

$x_{i}$ — the $i$ -th example input (e.g. a review).
$y_{i}$ — its output/label (e.g. negative).
${\dots}_{i = 1}^{k}$ — a collection of $k$ such pairs. $k = 0$ → zero-shot, $k = 1$ → one-shot, $k > 1$ → few-shot.
$x$ (no subscript) — the query from the preview: your real question, the one whose answer $y$ you actually want.

WHY. Each demo is more context, so by Step 3 it re-shapes the bars. But demos do something instructions can't: they show the exact output token pattern. The model, being a copier of patterns, mirrors that shape.

PICTURE. The same review now sits after two demos that each end in a single-word label. The chart's mass now sits almost entirely on the clean label token negative; the chatty-sentence bar has been crushed.

And the final answer is the tallest bar of that sharpened chart:

$answer = ar g max_{y} P_{θ} (y ∣ D, x)$

Here $ar g max_{y}$ means "the value of $y$ that makes the probability the biggest" — literally point at the tallest bar and read off its label.

Step 6 — When the answer is more than one token

WHAT. So far we drew $y$ as one clean bar (negative). But most answers — a JSON object, a sentence, a number like 30 split into pieces — are sequences of tokens $y = (y_{1}, y_{2}, \dots, y_{m})$ . We cannot read a single bar off a single chart; the model builds $y$ the same staircase way as Step 2.

WHY reuse the chain rule. The question "how likely is this whole answer?" is exactly the question the chain rule from Step 2 answers. We invoke it a second time, now inside the conditional, so that $P_{θ} (y ∣ D, x)$ still means something when $y$ has many tokens.

PICTURE. A mini-staircase after the demos: the context is $D$ then $x$ , and the model emits $y_{1}$ , folds it back, emits $y_{2}$ , and so on — each step its own bar chart, each shaped by the demos plus everything already written of the answer.

Step 7 — Edge case: the imbalanced-demos trap

WHAT. Demos don't only teach format — their label mix leaks into the chart. If most demos share one label, the model tilts toward copying that majority label, even on a query that deserves the other one.

WHY show this. The parent note warns "more examples ≠ better". This is the mechanism. Skipping it would leave you a scenario the pictures never covered.

PICTURE. Two demo sets, same query.

Balanced (top): equal positive/negative demos → the chart peaks on the correct label.
Imbalanced (bottom): four positive demos, zero negative → a bias arrow drags probability toward positive, and the model mislabels a clearly-negative query.

Step 8 — Edge case: order matters — recency bias

WHAT. The demos in $D$ are not weighted equally. Because the query sits right after the last demo, the model tends to lean hardest on the most recent examples — a tilt called recency bias (a form of demo-ordering sensitivity). Shuffle the same demos and the answer can change.

WHY it happens — straight from Step 2. Every answer token is conditioned on $x_{< t}$ , and the tokens closest to the prediction point are the last demos plus the query. Nearer context exerts a stronger pull, so the label pattern of the final demo is echoed most.

PICTURE. The same three demos in two orders. When the last demo is positive, the answer chart tilts positive; reorder so the last demo is negative, and the very same demo set now tilts negative — a "recency gradient" arrow points from the query back toward the nearest demo.

Step 9 — Edge case: the context window is a hard wall

WHAT. Every token of demos + query + answer must fit inside the model's context window — a fixed maximum number of tokens it can read at once.

WHY. From Step 2, each demo lengthens $x_{< t}$ . Lengthen it too far and the earliest tokens fall off the left edge — the model literally cannot see them. So $k$ can't grow forever, and accuracy plateaus well before the wall.

PICTURE. A fixed-width "window" box. As we stack more demos, the bar of used tokens grows rightward; past the wall the oldest demo is clipped — its contribution to the chart vanishes.

The one-picture summary

Everything above is one story: your prompt is the left half of every prediction, and its job is to turn a flat, ambiguous bar chart into a peaked, correct one — by conditioning, never by retraining.

The summary figure lays the whole pipeline in a row:

bare model → flat chart (unsure),
- instruction → narrower chart (right topic, loose shape),
- balanced demos (with the last one not lopsided) → sharp chart peaked on the clean label,
answers longer than one token are built token-by-token via the chain rule,
and the always-present guardrail: it must fit the context window.

Recall Feynman: the whole walkthrough in plain words

A language model is autocomplete that has read everything. When you ask it something, it doesn't answer with one word — inside, it builds a giant chart of "how likely is each possible next word", using its frozen brain-numbers (the weights) plus everything you typed. Whatever you type becomes part of "everything before", so your words tilt the chart. Zero-shot = you describe the task in words; the chart points roughly the right way but might come out in the wrong shape (a whole sentence instead of just "negative"). Few-shot = you first show a couple of finished examples (the demos) before your real question (the query); the model copies their shape, so the chart snaps onto one clean answer. If the answer is long, the model writes it one token at a time, each token another little chart. Crucially the brain-numbers never change — you only nudged the chart. Three traps: if your examples lean toward one label, the chart leans that way too (bias); the examples nearest the question count most, so their order matters (recency); and every example eats space in a fixed-size window, so you can't add infinitely many. Show a few, balanced, ending on a balanced note, in a consistent format — then get out of the way.

Recall Quick self-test

What does $ar g max_{y} P_{θ} (y ∣ D, x)$ tell you to do ::: Point at the tallest bar of the demo-sharpened chart and read off its label $y$ . Why does adding a demo change the output if weights don't change ::: The demo enters $x_{< t}$ , and the chart is computed from context, so more context = re-shaped chart. What three loose things do demos pin down ::: Output format, label space, and style/granularity. How is a multi-token answer's probability computed ::: By the chain rule again — a product of $P_{θ} (y_{j} ∣ D, x, y_{< j})$ over the answer's tokens. Why does demo order matter ::: Recency bias — demos nearest the query pull hardest, so the last examples carry more weight. Why can't you keep adding demos ::: They consume the fixed context window and give diminishing returns. What goes wrong with 4 positive + 0 negative demos ::: The distribution is biased toward the majority label; a negative query may be mislabeled positive.

Connections

Zero-shot and few-shot prompting — the parent topic this walkthrough visualizes.
Large Language Models — the next-token predictor whose bar chart we kept redrawing.
In-context learning — the name for chart-reshaping without weight change.
Chain-of-thought prompting — demos that also show reasoning steps, sharpening harder charts.
Prompt engineering — the craft of shaping context on purpose.
Retrieval-Augmented Generation (RAG) — auto-inserting retrieved text as extra grounding context.
Fine-tuning vs prompting — the alternative that does move $θ$ , dodging the window wall.
Context window — the hard token budget from Step 9.