In-context learning mechanisms: Exercises — AI-ML deep dive

This page tests the machinery from the parent note: the Bayesian task-inference view, the attention-as-one-GD-step toy model, and the empirical quirks (recency, label-space, saturation). Every problem has a collapsible full solution — try first, then reveal.

Before we start, we fix the vocabulary used throughout, so no symbol appears unexplained.

Level 1 — Recognition

L1.1 — Is this in-context learning?

Two students describe how they got an LLM to classify emails as spam/not-spam.

Student A: "I collected 5000 labelled emails and ran training that adjusted the network's weights until it stopped improving."
Student B: "I pasted 4 example emails with their labels into the prompt, then a 5th email, and read off the answer. I never touched the weights."

Which one is doing in-context learning? Name the single defining feature that decides it.

Recall Solution

Student B is doing ICL. The single defining feature is: the weights $θ$ never change — the task is learned purely by conditioning on demonstrations in the prompt during one forward pass. Student A is doing fine-tuning: the weights are permanently updated by gradient descent. Number of examples (5000 vs 4) is a red herring — the deciding line is weight update vs no weight update.

L1.2 — Name the shot count

Classify each prompt by its $k$ (the shot count):

Translate to French: "hello" → (no examples given, just the instruction and query)
dog → animal; then: rose → ?
A → 1; B → 2; C → 3; then: D → ?

Recall Solution

$k = 0$ → zero-shot (no demo pairs, only instruction + query).
$k = 1$ → one-shot (one demo dog → animal).
$k = 3$ → three-shot (three demos A→1, B→2, C→3). Rule: $k$ counts completed input–output pairs, not the query and not any instruction line.

Level 2 — Application

L2.1 — Compute a two-concept posterior

The model is unsure between two tasks after seeing demos:

$c_{1}$ = "sentiment" with prior $p (c_{1}) = 0.5$ ,
$c_{2}$ = "topic-labelling" with prior $p (c_{2}) = 0.5$ .

Two demos arrive. Under $c_{1}$ their joint likelihood is $p (D_{2} ∣ c_{1}) = 0.6 \times 0.6 = 0.36$ . Under $c_{2}$ it is $p (D_{2} ∣ c_{2}) = 0.2 \times 0.2 = 0.04$ . Compute the posterior $p (c_{1} ∣ D_{2})$ using Step 1 of the parent's derivation.

Recall Solution

Bayes with a flat prior: $p (c_{1} ∣ D_{2}) = \frac{p ( c _{1} ) p ( D _{2} ∣ c _{1} )}{p ( c _{1} ) p ( D _{2} ∣ c _{1} ) + p ( c _{2} ) p ( D _{2} ∣ c _{2} )} = \frac{0.5 \cdot 0.36}{0.5 \cdot 0.36 + 0.5 \cdot 0.04} .$ Numerator $= 0.18$ . Denominator $= 0.18 + 0.02 = 0.20$ . So $p (c_{1} ∣ D_{2}) = \frac{0.18}{0.20} = 0.9 .$ Two demos already push belief in the true task from $0.5$ to $0.9$ — that is the posterior beginning to collapse.

L2.2 — One gradient step by hand

Use the toy model $\overset{y}{^} = W x$ from the parent note, starting at $W_{0} = 0$ , learning rate $η = 1$ , with one demo $(x_{1}, y_{1})$ where $x_{1} = [10]$ and $y_{1} = 3$ (a scalar). Predict on the query $x_{2} = [20]$ .

Recall Solution

Where the update comes from (one-sentence reminder). We train the linear model by gradient descent on the squared-error loss $L = \frac{1}{2} \sum_{i} ∥ y_{i} - W x_{i} ∥^{2}$ . Its gradient is $\nabla_{W} L = - \sum_{i} (y_{i} - W x_{i}) x_{i}^{⊤}$ ; evaluated at the start $W_{0} = 0$ this is $- \sum_{i} y_{i} x_{i}^{⊤}$ , so one step $W_{1} = W_{0} - η \nabla_{W} L$ gives $W_{1} = η \sum_{i} y_{i} x_{i}^{⊤} .$ Step — build $W_{1}$ . With one demo: $W_{1} = η y_{1} x_{1}^{⊤} = 1 \cdot 3 \cdot [10] = [30] .$ Step — predict. $\overset{y}{^} = W_{1} x_{2} = [30] [20] = 3 \cdot 2 + 0 \cdot 0 = 6 .$ Cross-check via the "attention form." Substitute $W_{1} = η \sum_{i} y_{i} x_{i}^{⊤}$ back into $\overset{y}{^} = W_{1} x_{2}$ : $\overset{y}{^} = (η \sum_{i} y_{i} x_{i}^{⊤}) x_{2} = η \sum_{i} y_{i} (x_{i}^{⊤} x_{2}) .$ This is the same shape as a single (linear) attention head — it weights each demo answer $y_{i}$ by the similarity $x_{i}^{⊤} x_{2}$ between that demo's input and the query, exactly the equivalence the parent note derives. Numerically: $x_{1}^{⊤} x_{2} = 1 \cdot 2 + 0 \cdot 0 = 2$ , so $\overset{y}{^} = 3 \cdot 2 = 6$ . Identical to the matrix route. ✓

Level 3 — Analysis

L3.1 — Why does accuracy saturate?

The parent claims the log-posterior-ratio grows like $k \cdot KL$ . Suppose per demo the true concept beats a rival by $KL = ln 2 \approx 0.693$ nats (natural-log units, defined in the notation box), and start from equal priors. Give the posterior odds $\frac{p ( c ^{*} ∣ D _{k} )}{p ( c ^{'} ∣ D _{k} )}$ after $k = 1, 2, 3, 10$ demos, and explain in one sentence why gains "plateau".

Recall Solution

Log-odds accumulate. With two concepts, the log-odds is $ln \frac{p ( c ^{*} ∣ D _{k} )}{p ( c ^{'} ∣ D _{k} )}$ . From the parent's Step 3 it equals (equal priors ⇒ start at $0$ ) $ln \frac{p ( c ^{*} ∣ D _{k} )}{p ( c ^{'} ∣ D _{k} )} = 0 + \sum_{i = 1}^{k} KL = k ln 2.$ Exponentiate to get odds. Odds $= \frac{p ( c ^{*} ∣ D _{k} )}{p ( c ^{'} ∣ D _{k} )} = e^{k l n 2} = 2^{k}$ . Convert odds back to probability. For two mutually exclusive concepts $p (c^{'} ∣ D_{k}) = 1 - p (c^{*} ∣ D_{k})$ , so odds $= \frac{p}{1 - p}$ inverts to $p = \frac{odds}{odds + 1} = \frac{2 ^{k}}{2 ^{k} + 1}$ — this is exactly the sigmoid $σ (k ln 2)$ from the definition box.

$k$	odds $2^{k}$	$p (c^{*} ∣ D_{k}) = \frac{2 ^{k}}{2 ^{k} + 1}$
1	2	$0.6 \overline{6}$
2	4	$0.8$
3	8	$0.8 \overline{8}$
10	1024	$0.99902 \dots$
Why it plateaus (one sentence): the odds grow exponentially, but the probability $\frac{2 ^{k}}{2 ^{k} + 1}$ is squashed into $[0, 1]$ , so going $k = 3 \to 10$ moves accuracy only from $0.889$ to $0.999$ — the KL "budget" is nearly spent and extra demos barely help. The figure makes this S-curve explicit.

Reading the figure: the horizontal axis is the number of demos $k$ ; the vertical axis is the posterior probability $p (c^{*} ∣ D_{k}) = \frac{2 ^{k}}{2 ^{k} + 1}$ on the true task. The teal S-curve rises fast for small $k$ (orange arrow: big early gains) then flattens toward $1$ (teal arrow: diminishing returns). The dashed plum line marks the $0.95$ level; the dotted plum line shows the first $k$ that clears it — $k = 5$ , computed in L5.1.

L3.2 — Flipped labels

A student flips all labels in a 4-shot sentiment prompt (Positive↔Negative) and finds accuracy barely drops. They conclude "labels are useless." Critique this using the parent's label-space vs label-correctness distinction. When would flipping actually break the model?

Recall Solution

The demos carry three signals: (a) input distribution, (b) label space (the set {Positive, Negative} and the → format), (c) the exact input→label mapping. Flipping labels destroys only (c) but preserves (a) and (b). For an easy, high-prior task like sentiment, the model recovers the correct mapping from its pretraining prior, so it ignores the corrupted (c) and rides on (a)+(b) — hence the small drop. When flipping breaks it: for a novel/arbitrary task the prior cannot solve (e.g. cat → 7, dog → 3 with a made-up rule). There the model has no fallback mapping, so it must trust (c); corrupting the labels then genuinely wrecks accuracy. So "labels are useless" is false in general — it's a special case of strong priors.

Level 4 — Synthesis

L4.1 — Design a probe for "conditioning, not learning"

You want to demonstrate that ICL is transient conditioning (activations), not a real weight update. Design a two-part experiment and state, for each part, what result would confirm the conditioning hypothesis.

Recall Solution

Part 1 — Reversibility. Run the model on a $k$ -shot prompt, get answer $\overset{y}{^}_{A}$ . Then run the same query in a fresh prompt with no demos. If the model reverts to zero-shot behaviour (the demo-induced skill vanishes), the adaptation was transient → conditioning confirmed. A weight update would persist across prompts; conditioning does not. Part 2 — Ordering sensitivity. Permute the demo order and re-run. If predictions flip with order (recency/majority bias), the model is conditioning on token positions, not integrating examples into a stable learned function → conditioning confirmed. True GD on a fixed dataset is order-insensitive at convergence; ICL is not. Together: reversibility + order-sensitivity are signatures of conditioning inside a single forward pass, matching the parent's Story 2 caveat that the "virtual GD step" lives in activations only.

L4.2 — Reconcile the two stories

The parent gives Story 1 (Bayesian task location) and Story 2 (attention ≈ one GD step). Argue in a short paragraph that they are not competing theories, using the toy linear result $\overset{y}{^} = \sum_{i} y_{i} (x_{i}^{⊤} x_{k + 1})$ .

Recall Solution

Both stories describe the same computation from different angles. Story 2's attention output $\overset{y}{^} = \sum_{i} y_{i} (x_{i}^{⊤} x_{k + 1})$ is a similarity-weighted vote of the demo answers: query-like demos ( $x_{i}^{⊤} x_{k + 1}$ large) dominate. That weighting is exactly what Story 1 calls "the posterior selecting which examples/concept apply." The dot-product similarity plays the role of the per-demo likelihood; summing over demos plays the role of marginalising over evidence. So the forward pass simultaneously (i) implements a GD-like update (Story 2 mechanics) and (ii) behaves like Bayesian task-inference (Story 1 semantics). The model already contains the algorithm; the context selects and steers it — one process, two vocabularies.

Level 5 — Mastery

L5.1 — Derive the saturation curve closed-form

Assume equal priors between $c^{*}$ and one rival $c^{'}$ , and a constant per-demo expected log-likelihood gap $g = KL > 0$ nats. Derive $p (c^{*} ∣ D_{k})$ as a function of $k$ , show it is a sigmoid in $k$ , find its value at $k = 0$ , its limit as $k \to \infty$ , and the $k$ at which it first exceeds $0.95$ when $g = ln 2$ .

Recall Solution

Log-odds. From the parent's Step 3, log-odds $= ln \frac{p ( c ^{*} )}{p ( c ^{'} )} + \sum_{i = 1}^{k} g = 0 + k g$ (equal priors ⇒ first term $0$ ). Convert to probability. With two concepts, using the sigmoid $σ (z) = \frac{e ^{z}}{e ^{z} + 1}$ from the definition box with $z = k g$ : $p (c^{*} ∣ D_{k}) = σ (k g) = \frac{e ^{k g}}{e ^{k g} + 1}$ — a sigmoid in $k$ .

At $k = 0$ : $σ (0) = \frac{1}{2}$ (no evidence ⇒ 50/50). ✓
As $k \to \infty$ : $σ (k g) \to 1$ (posterior collapses onto $c^{*}$ ). ✓ Threshold. Need $σ (k g) > 0.95$ . Invert the sigmoid (take log-odds of both sides): $k g > ln \frac{0.95}{0.05} = ln 19 \approx 2.944$ . With $g = ln 2 \approx 0.693$ : $k > \frac{l n 19}{l n 2} = \frac{2.944}{0.693} \approx 4.25 \Rightarrow k = 5 .$ So five demos first push confidence past $0.95$ — and every demo after that is deep in the flat part of the sigmoid, giving the plateau you saw in L3.1.

L5.2 — Weighted-vote breakdown of the toy predictor

Given demos $(x_{1}, y_{1}) = ([10], 5)$ and $(x_{2}, y_{2}) = ([01], - 2)$ , and query $x_{3} = [31]$ , use $\overset{y}{^} = \sum_{i} y_{i} (x_{i}^{⊤} x_{3})$ to compute the prediction, and identify which demo dominates and why.

Recall Solution

Similarities: $x_{1}^{⊤} x_{3} = 1 \cdot 3 + 0 \cdot 1 = 3$ ; $x_{2}^{⊤} x_{3} = 0 \cdot 3 + 1 \cdot 1 = 1$ . Weighted vote: $\overset{y}{^} = y_{1} (x_{1}^{⊤} x_{3}) + y_{2} (x_{2}^{⊤} x_{3}) = 5 \cdot 3 + (- 2) \cdot 1 = 15 - 2 = 13 .$ Domination: demo 1 contributes $+ 15$ , demo 2 only $- 2$ . Demo 1 wins because the query $x_{3} = [31]$ points mostly along $x_{1}$ 's direction (its first coordinate, $3$ , is large), so its dot-product similarity is 3× that of demo 2. This is Story 1's "query-relevant demos dominate the posterior" and Story 2's "attention weights by key–query similarity" — the same number, two readings.

Recall One-line self-test

Every numeric answer on this page: $0.9$ , $6$ , odds table, $13$ , threshold $k = 5$ — all confirmed in the machine-check block. If your hand-work disagreed, re-open the matching solution.