In-context learning mechanisms: Visual walkthrough

We will build every symbol before using it. If you have never seen a probability, a logarithm, or the word "posterior," start at Step 1 and do not skip.

Step 0 — The words we need (read this first)

Before any math, three plain-English objects. Every later symbol is one of these.

That is the entire vocabulary. Everything below is these three things interacting.

Step 1 — The model starts out unsure (the prior)

WHAT. Before you show any example, a pretrained model already knows about thousands of possible tasks. It does not know which one you want. So it holds a spread-out belief across all of them.

WHY. During pretraining (see Transformers and Attention) the model read text from millions of different tasks. So "which task is this?" is genuinely uncertain at the start of a fresh prompt. This starting belief is called the prior, written $p (c)$ .

PICTURE. Look at the figure: each black bar is one candidate task; the height is the model's belief. The bars are all roughly the same short height — the model is spread thin. The red bar is the true task $c^{*}$ (the star means "the real one"). Right now it is not taller than its neighbours. That is the problem we are about to fix.

Step 2 — One demonstration is one piece of evidence

WHAT. You paste in the first demo $(x_{1}, y_{1})$ . We ask: how well does each candidate task explain this demo?

WHY. A demo is a fact about the world. Some tasks predict it easily; others would be shocked by it. The number "how likely is this demo, if task $c$ were true?" is the likelihood, written $p (x_{1}, y_{1} ∣ c)$ . The bar " $∣$ " reads "given" — given that the task is $c$ .

PICTURE. Same tasks as before, but now each bar's height is the likelihood — how unsurprised task $c$ is by the demo. The red bar ( $c^{*}$ , the true task) is tall, because the real rule generated this demo, so of course it explains it well. A wrong task $c^{'}$ that happens to also fit is medium; a totally-off task is near zero.

Step 3 — Bayes' rule: turn evidence into an updated belief

WHAT. We combine the prior (Step 1, "how much I believed each task before") with the likelihood (Step 2, "how well each task explains the demo") to get an updated belief, the posterior.

WHY this tool and not another? We need a rule that says how belief should change when a fact arrives. That rule — the unique consistent one — is Bayes' rule: multiply prior by likelihood, then rescale so the heights sum to 1 again. We use it because it is exactly "update your belief with evidence," which is precisely our question.

PICTURE. Three rows stacked. Top row = prior (flat, from Step 1). Middle row = likelihood (from Step 2). Bottom row = their product, then normalized: the red bar has grown taller and the wrong bars shrank. Evidence sharpened the belief. The vertical multiply-arrows show prior-height × likelihood-height feeding the bottom bar.

Step 4 — Compare only two tasks: form the ratio

WHAT. Watching all bars at once is busy. Let's track just one race: the true task $c^{*}$ versus one stubborn wrong task $c^{'}$ . We look at the ratio of their beliefs.

WHY a ratio? In the ratio $\frac{p ( c ^{*} ∣ D _{k} )}{p ( c ^{'} ∣ D _{k} )}$ both the numerator and denominator carry the same normalizer $Z$ from Step 3 — and since $Z$ is identical on top and bottom, it cancels exactly. We are left with something clean: prior-ratio times a product of likelihood-ratios. Ratios let us throw away the clutter and see the real competition.

PICTURE. A simple 1-vs-1 tug-of-war bar: a line with $c^{*}$ pulling right (red) and $c^{'}$ pulling left (black). Above the rope sits the fraction; the cancelled normalizer $Z$ is crossed out to show it disappears.

Step 5 — Take logs: multiplication becomes addition

WHAT. A product of $k$ tugs is hard to reason about. We apply the logarithm, written $lo g$ , which turns "multiply" into "add."

WHY the log, specifically? We want to see the growth with $k$ . Products grow in a way that's hard to eyeball; sums grow visibly and linearly. The one tool that converts $lo g (a \cdot b) = lo g a + lo g b$ is the logarithm. So $lo g$ is chosen precisely because it makes " $k$ tugs multiplied" become " $k$ tugs added," and a sum of $k$ similar things is obviously $\approx k \times (one thing)$ .

PICTURE. Left panel: $k$ little multiply-tugs stacked as a tall product (hard to read). Right panel, after taking $lo g$ : the same tugs laid end-to-end as equal-length steps climbing a staircase — each demo adds one step of the same height. The red running total marches upward.

Step 6 — Each step has a positive average height (the KL divergence)

WHAT. How tall is one staircase step, on average? Averaging the per-demo log-ratio over demos actually produced by the true task gives a quantity called the KL divergence, written $KL (c^{*} ∥ c^{'})$ .

WHY does this matter? The KL divergence is never negative, and it is strictly positive whenever the two tasks are actually distinguishable (i.e. they'd produce different demos). So on average every step climbs upward by $KL \geq 0$ . The staircase can only go up.

PICTURE. Two probability curves drawn over the same demo-space: red = what the true task $c^{*}$ produces, black = what the wrong task $c^{'}$ produces. The shaded gap between them is the KL divergence — the average step height. If the curves were identical (indistinguishable tasks) the gap is zero and the staircase is flat; the wider they differ, the taller each step.

Step 7 — The belief in the true task races to certainty (main result)

WHAT. Put Steps 5 and 6 together. The log-ratio climbs like $k \cdot KL$ . Undo the log ( $lo g$ 's partner is the exponential): the ratio itself blows up like $e^{k \cdot KL}$ , which sends the true task's belief $p (c^{*} ∣ D_{k})$ toward $1$ .

WHY. A quantity growing linearly inside a log means the raw quantity grows exponentially. Exponential growth of "true vs wrong" means wrong tasks are crushed to zero belief very fast, so the true task's belief saturates at $1$ .

PICTURE. A curve of $p (c^{*} ∣ D_{k})$ against number of demos $k$ : it shoots up steeply from the low prior, then flattens as it hugs the ceiling at $1$ (drawn in red). The steep part = "each early demo helps a lot." The flat part = "diminishing returns — the plateau."

Step 8 — Edge and degenerate cases (never hit a surprise)

Every scenario the reader might meet, each with the same picture-logic.

The one-picture summary

Everything above, compressed: the flat prior (left) gets hit by a staircase of equal positive steps (one per demo, average height $KL$ ), and the belief curve rockets to the ceiling and flattens.

Recall Feynman retelling — the whole walkthrough in plain words

Imagine a clever friend who has read everything, but when you start talking they don't yet know which game you want to play. That's the flat prior — lots of guesses, none confident (Step 1). But note: they can only guess a game they've heard of before — a game with zero prior is invisible to them. You show one solved example. Some games explain it well, most don't (Step 2). Your friend quietly bumps up their belief in the games that fit and lowers the rest — that's just common sense about evidence, which mathematicians call Bayes' rule (Step 3), and it works one demo at a time only because each demo is a fresh independent clue about the same game. To see who's winning, you watch the true game race one stubborn rival; the shared bookkeeping (the normalizer $Z$ ) cancels and you're left with a tug-of-war, one tug per example (Step 4). Because multiplying many tugs is hard to picture, you take a log and it becomes a staircase — each example adds one step (Step 5). And every step, on average, goes up, because the true game and a wrong game genuinely differ (that gap is the KL divergence, Step 6). Add up $k$ equal upward steps and the belief in the true game shoots to certainty — fast at first, then flat once it hits the ceiling (Step 7). Your friend never studied, never changed who they are — they just figured out the game from the clues you laid down. That is in-context learning.

Active recall

Why does taking a ratio of two tasks simplify the algebra?

The shared normalizer

Z

from Bayes' rule is identical on top and bottom, so it cancels — leaving prior-ratio times a product of per-demo likelihood-ratios.

What hidden assumption lets the fit to

k

demos become a product

\prod p (x_{i}, y_{i} ∣ c)

Conditional independence of the demos given the task — once

c

is fixed each demo is generated on its own, so joint probability factors into a product.

Why take the logarithm of that ratio, and which base?

Log turns the product of

k

likelihood-ratios into a sum of

k

terms (a linear staircase); we use natural log (base

e

) to match the KL divergence, though any base works up to a constant.

What does one staircase step represent on average, and what sign is it?

The KL divergence between the true task and the rival task; it is

\geq 0

and strictly positive when the tasks are distinguishable.

Which two conditions are required for

p (c^{*} ∣ D_{k}) \to 1

A strictly positive prior

p (c^{*}) > 0

AND a strictly positive

KL > 0

against every rival; if either fails the exponential guarantee is void.

What happens when the true task had zero prior,

p (c^{*}) = 0

Bayes multiplies prior by likelihood, so the posterior stays

0

forever — no number of demos can help; you'd need fine-tuning to install the capability.

Did any weight change during this whole process?

No —

θ

is frozen; only the belief over which task shifts inside the forward pass.