In-context learning mechanisms

WHAT is in-context learning?

WHY does this even exist? During pretraining the model sees enormous text where a pattern established earlier in a document predicts later tokens (lists, tables, translations, Q&A threads). To minimise next-token loss it is forced to become good at "spot the pattern in the context, then continue it." ICL is that skill reused at inference time.

HOW does it work? Two mechanistic stories

Both stories are compatible: the model already contains the algorithm; the context selects and steers it.

Derivation: why more demos help (Bayesian task inference)

Formula Posterior over the latent task concentrates with

$k$ Let $c$ be the hidden concept (the true task). Prior $p(c)$ from pretraining. Each demo $(x_i,y_i)$ is (approximately) generated by the true concept $c^*$ .

Step 1 — Bayes on the demos. Posterior after $k$ demos: $p(c \mid D_k) \;=\; \frac{p(c)\prod_{i=1}^{k} p(x_i,y_i\mid c)}{\sum_{c'} p(c')\prod_{i=1}^{k} p(x_i,y_i\mid c')}$ Why this step? Demos are the evidence; Bayes updates our belief about which concept generated them.

Step 2 — Predict the query by marginalizing. $p(y\mid x_{k+1},D_k) \;=\; \sum_{c} p(y\mid x_{k+1},c)\,p(c\mid D_k)$ Why this step? The model doesn't commit to one task; it averages predictions weighted by how likely each task is.

Step 3 — Concentration. Take logs of the ratio between the true $c^*$ and any wrong $c'$ : $\log\frac{p(c^*\mid D_k)}{p(c'\mid D_k)} = \log\frac{p(c^*)}{p(c')} + \sum_{i=1}^{k}\log\frac{p(x_i,y_i\mid c^*)}{p(x_i,y_i\mid c')}$ Each term in the sum is on average $+\,\mathrm{KL}\big(p(\cdot\mid c^*)\,\|\,p(\cdot\mid c')\big) \ge 0$ . Why this step? The expected per-demo log-likelihood gap is a KL divergence, which is non-negative and strictly positive for distinguishable tasks.

Conclusion. The log-ratio grows $\approx k\cdot \mathrm{KL}$ , so $p(c^*\mid D_k)\;\to\;1 \quad\text{exponentially in } k.$ The posterior collapses onto the true task — this is why accuracy climbs and then saturates with more shots.

Toy mechanism: attention as one gradient step

Formula Linear-attention ↔ gradient descent (sketch)

Consider a linear model $\hat y = W x$ . Ordinary least-squares gradient step from $W_0=0$ on demos $\{(x_i,y_i)\}$ with loss $\tfrac12\sum_i\|y_i - Wx_i\|^2$ : $W_1 = W_0 - \eta\,\nabla_W L = \eta\sum_{i=1}^{k} y_i x_i^\top$ Why this step? $\nabla_W \tfrac12\|y_i-Wx_i\|^2 = -(y_i-W_0x_i)x_i^\top = -y_i x_i^\top$ at $W_0=0$ .

Prediction on the query $x_{k+1}$ : $\hat y = W_1 x_{k+1} = \eta\sum_{i=1}^{k} y_i\,(x_i^\top x_{k+1})$ Now compare to attention. A single (linear) attention head outputs $\sum_i \text{value}_i \cdot \text{sim}(\text{key}_i,\text{query})$ . Set $\text{value}_i = y_i$ , $\text{key}_i = x_i$ , $\text{query}=x_{k+1}$ , similarity $=$ dot product: $\text{Attn} = \sum_{i=1}^{k} y_i\,(x_i^\top x_{k+1})$ Identical form. So the forward pass of attention over demos reproduces one GD step — the network "trains" a virtual model in its activations. That is the mechanistic meaning of "learning without weight updates."

Worked examples

Worked example Example 1 — Sentiment (few-shot)

Review: "Loved every minute." → Positive
Review: "Total waste of money." → Negative
Review: "The plot dragged on." → ???

Why it works: the two demos raise $p(c=\text{sentiment classification})$ ; the label format ("→ Positive/Negative") tells the model the output space. Model marginalizes → predicts Negative.

Worked example Example 3 — Recency & ordering

A→1, B→2, C→3, then B→?

Models are biased toward recent demos and toward majority label. Reordering demos can flip predictions — a symptom that ICL is conditioning, not clean learning.

Common mistakes

Recall Feynman: explain to a 12-year-old

Imagine a super-smart parrot that has read the whole internet. You don't retrain it. You just show it a few examples first: "cat→animal, rose→plant, dog→?" Because it has seen millions of little lists like this before, it figures out the game you're playing and finishes the pattern: "animal". It didn't study for a test — it just noticed the rule you were hinting at right there in front of it. That noticing-and-continuing is in-context learning.

Active recall

What is in-context learning?

A frozen LLM performing a new task by conditioning on demonstrations in the prompt, with no weight updates; the "learning" happens in the forward pass.

In ICL, what changes about the model's parameters?

Nothing —

\theta

stays fixed; only the transient activations depend on the context.

State the k-shot ICL objective.

\hat y=\arg\max_y P_\theta(y\mid x_1,y_1,\dots,x_k,y_k,x_{k+1})

Bayesian view: what do demonstrations do?

They shift the posterior

p(c\mid D_k)

over latent tasks, concentrating it on the true task.

Why does accuracy saturate with more shots?

Log-posterior-ratio grows

\approx k\cdot\mathrm{KL}

, so

p(c^*\mid D_k)\to1

exponentially; extra demos give diminishing returns.

Why does a single GD step equal linear attention?

GD from

W_0{=}0

gives

\hat y=\eta\sum_i y_i(x_i^\top x_{k+1})

, the same form as attention with keys

x_i

, values

y_i

, query

x_{k+1}

Why can flipped labels still give good ICL accuracy?

Demos mainly convey input distribution, label space, and format (task location); pretraining priors supply the mapping — until the task is novel/hard.

Zero-shot vs one-shot vs few-shot?

k=0,1,many demonstrations respectively.

Two biases that reveal ICL is conditioning not clean learning?

Recency bias (later demos weighted more) and majority-label bias.

ICL vs fine-tuning in one line?

Fine-tuning permanently edits weights; ICL is per-prompt, reversible conditioning.

Connections

Transformers and Attention — attention is the substrate that implements ICL.
Prompt Engineering — demo count, order, and format are the levers of ICL.
Chain-of-Thought Prompting — ICL extended with reasoning steps in the demos.
Retrieval-Augmented Generation (RAG) — retrieved passages act as context that steers the same conditioning mechanism.
Bayesian Inference — the task-posterior derivation.
Gradient Descent — the implicit-optimization interpretation.
Fine-tuning vs Prompting — when to change weights vs steer with context.

Concept Map

Hinglish (regional understanding)

Intuition Hinglish mein samjho

Dekho, in-context learning ka matlab hai: aap model ko train nahi kar rahe. Model ke weights bilkul frozen hain. Aap sirf prompt ke andar do-chaar examples (demonstrations) daal dete ho, aur model unko dekh kar "game samajh leta hai" aur pattern aage badha deta hai. Jaise "cat→animal, rose→plant, dog→?" — model bina kisi training ke bol dega "animal". Yeh sab ek hi forward pass ke andar ho raha hai.

Ab kyun hota hai? Pretraining ke time model ne internet par lakhon aisi lists, tables aur Q&A threads dekhe jahan pehle wala pattern baad ke tokens predict karta hai. Next-token loss kam karne ke liye model majboor tha ki "context mein pattern pehchano aur continue karo" — yehi skill inference time pe reuse hoti hai. Bayesian tareeke se sochein: model ke andar bahut saare latent tasks ka mixture hai, aur demos us posterior ko shift kar dete hain sahi task ki taraf. Jitne zyada saaf demos, utna posterior true task pe collapse — isliye accuracy pehle tez badhti hai, phir plateau ho jaati hai (exponential concentration, $p(c^*\mid D_k)\to1$ ).

Ek gehri baat: attention actually ek chhota gradient descent step jaisa kaam karta hai. Keys ko inputs $x_i$ , values ko labels $y_i$ , aur query ko test input maano — attention ka output $\sum_i y_i (x_i^\top x_{k+1})$ ban jaata hai, jo bilkul ek GD step ke barabar hai. Matlab model apne activations mein ek "virtual model" train kar leta hai, par asli weights kabhi nahi badalte.

Practical tip: labels galat ho jayein tab bhi ICL kaafi accha chalta hai, kyunki demos mostly format aur label-space batate hain — actual mapping pretraining prior se aa jaati hai. Lekin agar task naya/mushkil hai, tab correct labels zaroori ho jaate hain. Aur haan — demo ka order aur recency matter karta hai, majority label ki taraf bias hota hai. Isliye prompt design ek skill hai!

WHAT is in-context learning?

HOW does it work? Two mechanistic stories

Derivation: why more demos help (Bayesian task inference)

Toy mechanism: attention as one gradient step

Worked examples

Common mistakes

Active recall

Connections

Concept Map

Hinglish (regional understanding)

Go deeper — visual, from zero

Test yourself — Alignment, Prompting & RAG

Connections