Foundations — Chain-of-thought prompting

This page builds every symbol, letter, and idea the parent note Chain-of-Thought Prompting leans on — starting from "what is a language model even doing" and ending with the voting formula. Nothing is assumed. If you meet a symbol below that you've seen before, good; if not, we build it here.

0. What is a language model, in pictures?

Before any symbol, the object itself.

Think of it as a very well-read autocomplete. You type "The capital of France is" and it leans toward "Paris".

Look at Figure 1: text flows in on the left, the model is the box in the middle, and one new word pops out on the right. That single-word-at-a-time habit is the whole reason Chain-of-Thought matters — hold onto it.

Why do we even need this word? Because the entire argument for CoT is measured in tokens: more tokens written = more thinking done. We'll make that precise in §4.

1. $q$, $a$, and the arrow $\to$

The parent's very first line writes $q\to a$. Let's earn each piece.

The picture: $q$ is what goes into the box in Figure 1; $a$ is (eventually) what comes out. Standard prompting is the naive hope that the box jumps straight from $q$ to $a$ in one leap.

2. The reasoning steps $r_1,r_2,\dots ,r_k$

Now the star of the show.

The picture: this is the scratch paper. Each $r_i$ is one scribbled line before you circle the final answer.

Compare the two rows in Figure 2. Top row (standard prompting): one giant, terrifying jump $q\to a$. Bottom row (Chain-of-Thought): the same distance, but broken into small hops $q\to r_1\to r_2\to \cdots \to r_k\to a$. Same start, same finish — but each hop is short enough to actually land.

3. Few-shot vs zero-shot: the word "shot"

The parent splits CoT into few-shot and zero-shot. That word "shot" needs a picture.

The picture: few-shot is handing the model a couple of answered homework problems as a template before giving it the new one. Zero-shot is handing it the new problem with only the instruction "show your working."

This idea comes straight from Few-shot In-context Learning — the model learns the format purely from examples inside the prompt, without any retraining. CoT is that trick, but the examples include reasoning, not just answers.

4. "Fixed computation per token" — the engine argument

This is the deepest assumption in the parent, and it uses no formula — so let's give it one clear picture.

Now the payoff: each reasoning token the model writes is fed back in as new input. So writing $r_1$ buys another forward pass to write $r_2$, which buys another for $r_3$…

Read Figure 3 as a staircase. Standard prompting stands at the bottom and must jump to the answer in one step (the huge dashed arrow). CoT climbs — each stair is one token, one forward pass, one small amount of new thinking that the next stair gets to build on. Same height reached, but by climbing instead of leaping.

This is why the tool "write intermediate tokens" is the right tool: the bottleneck was compute-per-answer, and writing tokens is the only knob that adds compute at inference time without retraining.

5. The probability symbols: $p(a\mid q)$ and the sum ${\sum }_r$

The parent's key equation is $p(a\mid q)={\sum }_{r}p(a\mid r,q)p(r\mid q)$. Every piece, from zero.

The picture: $p(a\mid q)$ is the model's confidence in an answer after reading the question but before writing any working. $p(a\mid r,q)$ is its confidence after it has already written the full chain of working — much sharper, because the hard part is done on paper.

Why this tool — a sum? Because there isn't one correct chain of working; there are many roads to the same answer. To get the total chance of landing on answer $a$, you must collect the probability from every road, and "collect over all cases" is exactly what a sum does.

Figure 4 shows the question node splitting into several complete reasoning paths (violet), each carrying a bit of probability, all of them pouring into the same answer node. The total arriving at that answer is the sum. A correct answer (right side) collects from many fat paths; a wrong answer collects from a few thin, scattered ones — which is the whole reason voting works.

6. Self-consistency symbols: $\hat{a}$, $\arg \max$, $1[\cdot ]$, $N$

The parent's voting formula is $\hat{a}=\arg {\max }_a{\sum }_{i=1}^{N}1[a_i=a]$. Let's decode it fully.

Put together in plain words: "Count the votes for each possible answer; return the answer with the most votes." That's it — the intimidating formula is just majority vote, exactly as in Self-Consistency Decoding.

7. How it all connects

Follow the arrows: the humble "one token = one forward pass" fact feeds the whole idea, and the sum-over-paths feeds both the why of CoT and the voting of self-consistency. Everything on this page is a prerequisite for the parent note Chain-of-Thought Prompting, and it opens the door to Tree of Thoughts, ReAct, and Emergent Abilities of LLMs.

Equipment checklist

Cover the right side and answer each before moving on.