Noesis
4.4.7Alignment, Prompting & RAG

Chain-of-thought prompting

2,022 words9 min readdifficulty · medium6 backlinks
🇮🇳 Yeh note Hinglish mein padho →

WHAT is Chain-of-Thought (CoT)?

The reasoning steps are generated in natural language, and the answer is read off from the tail of the generation.

Two main flavours

WHY does it work? (First-principles reasoning)

A transformer does a fixed amount of computation per generated token (one forward pass). For a hard problem, a single token of output is a tiny compute budget — the model must compress the entire multi-step reasoning into one forward pass, which it cannot do reliably.

Formally, think of the model as approximating p(a∣q)p(a \mid q). Marginalising over reasoning paths rr:

p(a∣q)=∑rp(a∣r,q) p(r∣q)p(a \mid q) = \sum_{r} p(a \mid r, q)\, p(r \mid q)
Figure — Chain-of-thought prompting

HOW to write a CoT prompt

Few-shot CoT template

Q: Roger has 5 tennis balls. He buys 2 cans of 3 balls each.
   How many balls does he have now?
A: Roger starts with 5 balls. 2 cans × 3 balls = 6 balls.
   5 + 6 = 11. The answer is 11.

Q: <your new question>
A:

Why this step? The exemplar demonstrates the format: reason first, then "The answer is X". The model pattern-matches and reproduces that structure.

Zero-shot CoT template

Q: <your question>
A: Let's think step by step.

Why this step? The trigger phrase shifts the model into a distribution where the next tokens are reasoning, not a bare answer.

Self-Consistency (the power-up)

Worked examples

Common mistakes

Recall Feynman: explain to a 12-year-old

Imagine a hard maths puzzle. If I force you to shout the answer instantly, you'll probably guess wrong. But if I let you write your working on scrap paper first — "okay, 23 apples, take away 20, that's 3, add 6, that's 9" — you'll get it right way more often. Chain-of-thought is just handing the AI a scrap paper and saying "show your working before you answer." And if you ask three friends to do the working separately and they mostly say "9", you trust 9 — that's self-consistency.

Flashcards

What is chain-of-thought prompting?
Guiding an LLM to produce intermediate reasoning steps before the final answer, i.e. q→r1..rk→aq\to r_1..r_k\to a instead of q→aq\to a.
Zero-shot CoT trigger phrase?
"Let's think step by step."
Difference between few-shot and zero-shot CoT?
Few-shot supplies worked exemplars in the prompt; zero-shot uses only a trigger phrase, no exemplars.
Why do reasoning tokens improve performance (compute view)?
Each generated token is a forward pass fed back into context, so CoT buys more effective computation and externalises intermediate results.
Probabilistic decomposition CoT exploits?
p(a∣q)=∑rp(a∣r,q) p(r∣q)p(a\mid q)=\sum_r p(a\mid r,q)\,p(r\mid q) — replacing one hard prediction with many easy, in-distribution ones.
What is self-consistency decoding?
Sample N diverse chains and take a majority vote over their final answers.
Why does majority voting work?
Correct answers are reached by many valid paths; errors are scattered, so voting concentrates on the most-supported answer.
When does CoT mainly help (Steel-man mistake)?
Emergently at large scale; small models may not benefit and can get worse.
What is the faithfulness problem in CoT?
The written chain may be a post-hoc rationalisation, not the true cause of the answer; chain correctness ≠ answer correctness.
Self-consistency formula?
a^=arg⁡max⁡a∑i1[ai=a]\hat a=\arg\max_a \sum_i \mathbb 1[a_i=a].

Connections

  • Prompt Engineering — CoT is a core prompting pattern.
  • Few-shot In-context Learning — few-shot CoT is a specialisation of ICL.
  • Self-Consistency Decoding — the sampling+vote upgrade to CoT.
  • Tree of Thoughts / ReAct — extend linear chains to search / tool-use.
  • Emergent Abilities of LLMs — explains why CoT needs scale.
  • Retrieval-Augmented Generation — combine external facts with CoT reasoning.
  • Temperature and Sampling — needed to generate diverse chains.

Concept Map

limited on hard tasks

generates

then read off

flavour

flavour

uses

uses exemplars

motivates

buys compute for

formalised by

calibrates

Standard prompting q to a

Chain-of-Thought

Intermediate reasoning steps

Final answer a

Few-shot CoT

Zero-shot CoT

Think step by step phrase

Fixed compute per token

Tokens as compute budget

Marginalise over paths r

Pretraining on step-by-step text

Hinglish (regional understanding)

Intuition Hinglish mein samjho

Chain-of-thought ka simple funda ye hai: model ko seedha answer maangne ke bajaye, hum bolte hain "step by step socho aur apna working likho." Kyun? Kyunki ek transformer har token ke liye sirf ek forward pass karta hai — matlab ek fixed compute budget. Agar hard problem ka answer ek hi token me nikaalna pade, to model ke paas sochne ki jagah hi nahi bachti. Har reasoning token likhne se model ko extra compute milta hai aur wo partial results scrap paper ki tarah likh ke rakh leta hai. Isi liye maths aur logic wale sawaalon me CoT ka accuracy kaafi badh jaata hai.

Do tareeke hain. Few-shot CoT me aap prompt me 2-3 solved examples dete ho jisme reasoning bhi dikhaya gaya ho — model pattern copy kar leta hai. Zero-shot CoT me bas ek jaadu wala phrase "Let's think step by step" laga do, examples ki zaroorat nahi. Dono ka goal same hai: model ko reasoning-mode me daalna.

Ek powerful upgrade hai self-consistency: ek hi sawaal ke liye thodi temperature ke saath kai alag-alag chains sample karo, har chain ka final answer nikaalo, aur majority vote le lo. Logic ye hai ki sahi answer tak pahunchne ke bahut saare valid raaste hote hain, par galtiyaan bikhri hui hoti hain — to vote karne se sahi answer upar aa jaata hai.

Ek important baat yaad rakho (exam trap): CoT mainly bade models me kaam karta hai — chhote models me kabhi kabhi ulta bigaad bhi deta hai. Aur jo chain model likhta hai wo hamesha "sacchi" wajah nahi hoti — kabhi kabhi model bas plausible bahaana bana deta hai (faithfulness problem). To chain sahi dikhe iska matlab answer bhi sahi ho, ye zaroori nahi.

Go deeper — visual, from zero

D1FoundationsEvery symbol from zero · 2,871 wordsD2Visual walkthroughThe derivation in pictures · 4,021 wordsD3Worked examplesEvery scenario covered · 3,007 wordsD4ExercisesGraded, full solutions · 2,542 wordsD5Question bankConcept traps, quick reveals · 1,408 words

Test yourself — Alignment, Prompting & RAG

L1 RecognitionL2 RecallL3 ProductionL4 ApplicationL5 Mastery

Connections

Prompt engineering best practicesAI-ML · 4.4.10Retrieval-Augmented Generation (RAG) architectureAI-ML · 4.4.11Zero-shot and few-shot promptingAI-ML · 4.4.6Self-consistency and tree-of-thoughtAI-ML · 4.4.8In-context learning mechanismsAI-ML · 4.4.9Prompt engineering best practicesAI-ML · 4.4.10Hallucination mitigationAI-ML · 4.4.15Emergent abilities in large modelsAI-ML · 6.1.3