4.4.7 · D5Alignment, Prompting & RAG
Question bank — Chain-of-thought prompting
Before you start, three words to keep in your head from the parent note:
- A reasoning step () is an intermediate token the model writes on its "scratchpad" before the final answer .
- Few-shot CoT shows worked exemplars in the prompt; zero-shot CoT just adds a trigger like "Let's think step by step."
- Self-consistency samples many chains and majority-votes the answer.
True or false — justify
CoT changes the model's weights so it permanently gets better at reasoning.
False. CoT is only a prompt — it steers which tokens come next at inference time; no gradient step is taken and no weight changes. It unlocks a latent ability, it doesn't train one.
Zero-shot CoT works because "Let's think step by step" is a hardcoded keyword the model was trained to detect.
False. The phrase just shifts the output distribution toward reasoning text; many paraphrases ("First, let's...", "Working it out:") work too. See Prompt Engineering — it's steering, not a magic switch.
A longer chain of reasoning always means a more correct answer.
False. Longer chains can drift and accumulate errors, and a fluent chain can still be a post-hoc rationalisation. Chain length and answer correctness are only loosely coupled.
If the final answer is correct, the reasoning chain that produced it must also be correct.
False. This is the faithfulness problem: the model can reach a right answer via a chain full of wrong or irrelevant steps, or a chain that isn't the true cause of the answer at all.
CoT helps models of every size, since prompting is model-agnostic.
False. Gains are largely emergent with scale (see Emergent Abilities of LLMs); small models can get worse, producing confident wrong chains. The ability must already exist latently for CoT to surface it.
Self-consistency requires temperature 0 (greedy decoding).
False. It needs temperature > 0 so the sampled chains are diverse; identical greedy chains would all vote the same way, giving no benefit. See Temperature and Sampling.
The probabilistic view is just notation and adds nothing operationally.
False. It's the justification for both CoT (sample one , making easier and in-distribution) and self-consistency (average over many to estimate the marginal). The decomposition is the recipe.
Few-shot CoT and few-shot in-context learning are the same thing.
False. Few-shot In-context Learning shows input→output pairs; few-shot CoT shows input→reasoning→output. The extra reasoning in each exemplar is exactly what teaches the format.
Spot the error
"CoT gives the model more memory by increasing its hidden state size."
The hidden state size is fixed by architecture. What CoT adds is externalised memory — partial results are written into the context tokens, which are re-read on later forward passes, not stored in a bigger hidden state.
"Each reasoning token is free — it costs nothing extra to generate a long chain."
Wrong on cost. Every token is a full forward pass, so a long chain costs more compute and latency. The benefit is buying that compute deliberately; it is not free.
"Self-consistency works by picking the longest, most detailed chain."
No — it picks the most-agreed-upon final answer via majority vote, ignoring chain length entirely. A short chain and a long chain that reach the same answer count as one vote each.
"To do zero-shot CoT you must also include a worked example."
Contradiction — if you include a worked example it's few-shot CoT. Zero-shot means the trigger phrase and nothing else.
"CoT works because natural-language reasoning is inherently more logical than a single-token answer."
The mechanism isn't logic-in-language; it's that pretraining data is full of step-by-step human text, so is well-calibrated in-distribution when is present. It's about matching training patterns plus more compute.
"If two sampled chains disagree, self-consistency has failed."
Disagreement is expected and is the whole point — the method tolerates scattered errors and lets the correct answer, reachable by many valid paths, win the vote. Disagreement only fails it if there is no majority.
Why questions
Why is asking for the answer directly () a "tiny compute budget" for a hard problem?
Because a transformer runs a fixed amount of computation per generated token; one output token = one forward pass, forcing all multi-step reasoning to be compressed into that single pass.
Why does writing intermediate results down help even when the model "knows" them?
The written tokens are fed back into context, so the model doesn't have to hold every partial result in one hidden state — it offloads memory and reads its own work back later.
Why does self-consistency concentrate probability on the correct answer rather than a wrong one?
Correct answers are reachable by many independent valid reasoning paths, so their votes pile up; wrong answers come from idiosyncratic, scattered errors that rarely coincide, so no single wrong answer accumulates a majority.
Why might we prefer Self-Consistency Decoding over just lowering the temperature to get one confident chain?
A single confident chain can be confidently wrong; sampling many diverse chains and voting is a Monte-Carlo estimate of the marginal that averages out individual mistakes.
Why is "closer to patterns seen during pretraining" than for a hard task?
Pretraining text usually shows its working, so predicting an answer given its reasoning is on-distribution; predicting an answer with the reasoning hidden is off-distribution for hard problems the model can't do in one shot.
Why does CoT relate to more advanced methods like Tree of Thoughts and ReAct?
They generalise the same idea — spend tokens on intermediate structure before answering: ToT explores a branching search over reasoning steps, ReAct interleaves reasoning with actions/tool-calls. CoT is the linear special case.
Edge cases
For a trivial one-step question (e.g. "What is 2+2?"), does CoT help?
Little to none — there's no multi-step computation to spread out, and forcing reasoning can even add noise. CoT's benefit scales with problem depth, not with every query.
What happens if you apply self-consistency but the task has no single extractable final answer (e.g. an open essay)?
Majority voting breaks down — there's nothing discrete to count votes over. Self-consistency assumes answers are comparable/extractable, so it fits arithmetic, multiple-choice, or short-answer tasks.
What if all sampled chains give different answers with no majority?
The vote is undefined or ties; self-consistency gives no clear winner. In practice this signals the model is highly uncertain, and larger or a stronger model is needed — voting can't manufacture agreement that isn't there.
If a small model produces a fluent, well-structured chain, can we trust the answer?
Not on fluency alone — small models are precisely where CoT can produce fluent-but-wrong chains, since the reasoning ability is below the emergence threshold. Structure is not evidence of correctness.
At temperature 0, what does self-consistency degenerate into?
A single deterministic chain (all samples identical), so majority vote returns that one answer — it collapses back to plain greedy CoT with no diversity benefit.
Does combining CoT with Retrieval-Augmented Generation change what the reasoning tokens do?
The reasoning can now cite and chain over retrieved facts rather than only the model's parametric memory, which helps knowledge-heavy multi-step questions — but the faithfulness caveat still holds: retrieved-and-reasoned isn't guaranteed to be the true causal path.
Recall One-line summary of the traps
CoT and self-consistency spend tokens to buy compute and average out errors — but they don't retrain the model, don't guarantee faithful reasoning, and mainly help large models on multi-step problems with extractable answers.