5.3.11 · D1Advanced Microarchitecture

Foundations — Speculative execution

2,040 words9 min readBack to topic

Before you can read the parent note, you need to own a small pile of words and pictures. This page builds each one from nothing, in an order where every idea leans only on the ones before it. Nothing here assumes you already know what a "pipeline" or a "register" is.


1. The instruction — the atom of everything

The picture: think of a recipe. Each numbered line ("crack an egg", "stir") is one instruction. The CPU is the cook working down the list.

Why the topic needs it: speculation is about doing future recipe lines early, before you're sure you'll need them. You can't talk about "running ahead" without first agreeing that a program is an ordered list of atoms.


2. Registers, memory, flags — the CPU's visible worktop

The picture: a kitchen worktop (registers = the few bowls in front of you), a pantry (memory), and a little status light (flag). The parent note calls all three together the architectural state — the stuff a program is allowed to see.

Why the topic needs it: the whole trick of speculation is changing these visibly only when you're sure. So you must first know what "the visible stuff" even is.


3. What the >, ==, and * symbols mean here


4. The branch — the fork in the road

Figure — Speculative execution

The picture (figure above): a single track that splits into two. The pale-yellow diamond is the branch instruction; blue is the "taken" path, pink the "not-taken". The CPU is a train that must choose before it knows which track is correct.

Why the topic needs it: branches are the reason speculation exists. If code never forked, the CPU would always know what comes next and never need to guess. See Pipeline hazards for why a fork specifically causes a stall.


5. The pipeline — why waiting hurts

Figure — Speculative execution

The picture (figure above): a car wash. At any moment one car is being soaped, another rinsed, another dried. The parent says pipelines are "15–20+ stages" — that means 15–20 cars in the wash at once.

Why the topic needs it: the entire payoff of speculation is filling bubbles. The deeper the pipeline, the more bubbles a stalled branch creates, and the more speculation saves. This directly builds on Instruction-level parallelism (ILP) — running several instructions at once — and Superscalar architecture, which widens the line to more than one car per stage.


6. The clock cycle — the unit we measure cost in


7. Prediction, checkpoint, verify, commit, squash

Now the five verbs of speculation. Each one only makes sense once you have branches, a pipeline, and cycles.

Figure — Speculative execution

The picture (figure above): two timelines. Top (correct guess): the CPU races ahead on the blue path and everything commits — no wasted time. Bottom (wrong guess): the CPU races down blue, discovers pink was correct, squashes the blue work (the crossed-out block), and refetches — the yellow gap is the penalty.


8. The scratchpad: architectural vs. microarchitectural state

This is the single most important distinction in the whole topic.

The picture: architectural state is the final published essay; microarchitectural state is your messy draft notebook. You can erase the notebook (squash) with nobody the wiser — almost. The parent's Spectre section is the twist: the notebook leaves faint smudges (cache timing) an attacker can read. See Cache side-channel attacks.

Why the topic needs it: without this two-layer picture, "undoing" wrong work sounds impossible, and Spectre sounds like magic. With it, both are obvious: undo the visible layer, but the private layer keeps a smudge.

This layer also connects to Out-of-order execution — instructions execute whenever their inputs are ready, not in list order — and the ROB is what lets the CPU reorder execution yet still commit in order.


9. Reading the parent's formulas


The prerequisite map

Instruction (the atom)

Branch (the fork)

Registers Memory Flags

Architectural state (visible)

Pipeline (assembly line)

Pipeline bubble and stall

Cycle (unit of cost)

Microarch state (scratchpad)

Reorder Buffer ROB

Cache

Predict verify commit squash

Speculative Execution


Equipment checklist

Test yourself — each line hides the answer.

What is an instruction, in one phrase?
One tiny order the CPU obeys; a program is an ordered list of them.
Name the three parts of architectural state.
Registers, memory, and flags — the state software is allowed to see.
What is the difference between = and ==?
= stores a value; == asks a yes/no equality question.
What is a branch and why does it force a stall?
A fork where the CPU must pick a road; until the condition resolves, fetch doesn't know which instructions to grab, causing bubbles.
What does pipeline depth mean, and why does bigger make mispredictions costlier?
The number of overlapped stages/in-flight instructions; a wrong guess squashes up to of them, so more work is wasted.
What is a pipeline bubble?
An idle wasted cycle where a stage waits for missing information.
In what unit is every speculation cost measured?
CPU clock cycles.
List the five verbs of speculation in order.
Predict, checkpoint, verify, commit (if right) or squash (if wrong).
Where do speculative results wait before committing?
In the Reorder Buffer (ROB), part of microarchitectural state.
What is the core difference between architectural and microarchitectural state?
Architectural = the visible promise (registers/memory); microarchitectural = private scratchpads/caches, updated immediately and (mostly) erasable on squash.
Why is squashing not perfectly invisible?
Speculative loads can leave data in the cache; timing that cache leaks the secret (Spectre / side channels).
Fill in the penalty formula.
Penalty (cycles) = Pipeline Depth + Refetch Latency.