In-context learning mechanisms: Foundations — AI-ML deep dive

This page assumes you have seen nothing. We define every letter, arrow, and squiggle the parent note (In-context learning mechanisms) throws at you, in an order where each one only uses things already built.

1. What is a "probability" $p (\cdot)$ ?

Figure s01 — a length-1 bar cut into coloured slices (Negative 0.55, Positive 0.30, Neutral 0.15); the width of each slice is its probability and the slices exactly fill the bar. Refer to this bar every time the word "probability" appears below.

Why the topic needs it. An LLM never says "the answer IS Negative." It says "Negative fills 0.8 of the bar, Positive fills 0.2." Every formula in the parent note is really about widths of that bar.

2. Conditional probability — the bar $p (y ∣ x)$

The vertical stroke $∣$ is read "given". It is the single most important symbol on the parent page.

Figure s02 — left: the full bar over four outcomes A,B,C,D; right: after we are "given $x$ ", only the lit outcomes B and C survive and their widths are re-measured so they again fill length 1 (each becomes 0.5). This is the picture behind the division in the formula below.

Why the topic needs it. The whole model is a conditional-probability machine: $P_{θ} (y ∣ context)$ = "how likely is the next chunk $y$ , given everything typed so far." The demonstrations just make the "given" part longer. (The capital $P$ here is explained in the very next section.)

3. The subscript $θ$ — "the model's settings", and why $P$ is capital

4. $x$ , $y$ , and pairs $(x_{i}, y_{i})$ — inputs, outputs, demonstrations

Why the topic needs it. These pairs are the raw material. In §7 we will count how many of them we have, and in §8 use the "widest bar" idea to pick an answer — but first we need those two ideas as their own symbols. For now just remember: a prompt is some demonstrations, then one query with a missing back.

5. $k$ and " $k$ -shot"

6. $ar g max$ — "which choice wins?"

The hat in $\overset{y}{^}$ means "the model's guess for $y$ ", to distinguish it from the true answer.

7. Sum $\sum$ and product $\prod$

8. Latent task / concept $c$

9. Prior $p (c)$ , posterior $p (c ∣ D_{k})$ , and Bayes' rule

Definition Prior, evidence, posterior, likelihood

$D_{k} = {(x_{1}, y_{1}), \dots, (x_{k}, y_{k})}$ — the bag of $k$ demonstrations (the evidence).
$p (c)$ — the prior: belief about the task before seeing any demos, baked in by pretraining.
$p (c ∣ D_{k})$ — the posterior: updated belief after reading the demos.
$p (D_{k} ∣ c)$ — the likelihood: how well rulebook $c$ explains the whole bag of demos. If each demonstration is generated independently once the task $c$ is fixed, the whole-bag probability is the product (§7) of the single-demo probabilities: $p (D_{k} ∣ c) = \prod_{i = 1}^{k} p (x_{i}, y_{i} ∣ c) .$ WHY a product: under the independence assumption, joint probability of many independent events = multiply their individual probabilities; a rulebook that fits every demo well gets a large product, one that stumbles on any demo gets dragged toward $0$ .

Why the topic needs it. The parent's key claim — "accuracy climbs then saturates" — is literally "the posterior bar collapses onto one rulebook and can't get any narrower." See Retrieval-Augmented Generation (RAG) for a different way of feeding evidence — by fetching text instead of squeezing a prior.

10. KL divergence $KL (p ∥ q)$

Formula Its formula, its assumptions, and why it is never negative

For two bars $p$ and $q$ over the same outcomes, $KL (p ∥ q) = \sum_{z} p (z) lo g \frac{p ( z )}{q ( z )} .$ Assumptions needed: both are proper length-1 bars ( $\sum_{z} p (z) = \sum_{z} q (z) = 1$ ), and $q (z) > 0$ wherever $p (z) > 0$ (otherwise we would divide by $0$ / take $lo g$ of $\infty$ ). Terms with $p (z) = 0$ contribute $0$ by convention. WHY it is $\geq 0$ — step by step. Look at the negated quantity and flip the ratio inside the log: $- KL (p ∥ q) = \sum_{z} p (z) lo g \frac{q ( z )}{p ( z )} .$ The function $lo g$ is concave (it bends downward), so Jensen's inequality — "the average of $lo g$ is at most $lo g$ of the average" — gives $\sum_{z} p (z) lo g \frac{q ( z )}{p ( z )} \leq lo g (\sum_{z} p (z) \frac{q ( z )}{p ( z )}) = lo g (\sum_{z} q (z)) = lo g 1 = 0.$ So $- KL (p ∥ q) \leq 0$ ; multiply both sides by $- 1$ (which flips the inequality) to get $KL (p ∥ q) \geq 0$ . Equality holds only when $q = p$ everywhere.

11. Vectors, dot products $x^{⊤} x^{'}$ , and $W$

The "attention ≈ one gradient step" section needs three tiny linear-algebra pieces. To avoid a clash with the discrete label $\overset{y}{^}$ from §6, this section writes the vector output of a linear map as $o$ ("output vector").

Figure s03 — an amber "query" arrow and three "key" arrows: one aligned (dot product > 0), one perpendicular (dot ≈ 0), one opposite (dot < 0). The dot product number beside each key is the "similarity meter" reading attention uses to weight that key.

p (y ∣ x)

means.

"The probability of

y

given that

x

is already known"; we restrict to outcomes agreeing with

x

and re-measure widths to sum to

1

I can write the formal relation for conditional probability.

p (y ∣ x) = p (x, y) / p (x)

— joint over marginal; dividing by

p (x)

re-normalises the lit region to length

1

I can explain the difference between lowercase

p

and capital

P_{θ}

p

is the true/idealised probability in the world;

P_{θ}

is the model's own estimate computed from its weights

θ

— same kind of object, but an approximation.

I know what

θ

is and whether ICL changes it.

θ

is the full list of the model's weights (the sliders); ICL leaves it frozen — only the input text changes.

I can read

ar g max_{y} f (y)

out loud.

"The choice of

y

that makes

f

biggest" — it returns the argument (widest bar), not the value.

I can expand

\sum_{i = 1}^{k} a_{i}

and

\prod_{i = 1}^{k} a_{i}

a_{1} + \dots + a_{k}

(add) and

a_{1} \times \dots \times a_{k}

(multiply).

I can write the likelihood of the whole demo bag under a task

c

p (D_{k} ∣ c) = \prod_{i = 1}^{k} p (x_{i}, y_{i} ∣ c)

under the assumption demos are independent given

c

I can state Bayes' rule for the task posterior.

p (c ∣ D_{k}) = p (D_{k} ∣ c) p (c) / p (D_{k})

with

p (D_{k}) = \sum_{c^{'}} p (D_{k} ∣ c^{'}) p (c^{'})

I can write the marginalization used to answer the query.

p (y ∣ x_{k + 1}, D_{k}) = \sum_{c} p (y ∣ x_{k + 1}, c) p (c ∣ D_{k})

— average each task's answer by its posterior weight.

I can write the KL formula and say why it is

\geq 0

KL (p ∥ q) = \sum_{z} p (z) lo g \frac{p ( z )}{q ( z )}

; requires

q (z) > 0

wherever

p (z) > 0

; non-negative by concavity of

lo g

(Jensen), zero iff

p = q

I can explain what a dot product

x^{⊤} x^{'}

measures.

How aligned two vectors are: large positive = same direction,

0

= perpendicular.

I know what a gradient

\nabla_{W} L

and step size

η

do.

Gradient points uphill on the loss landscape; step

- η \nabla_{W} L

walks downhill by amount

η

Foundations — In-context learning mechanisms