5.3.11 · D5Advanced Microarchitecture

Question bank — Speculative execution

1,541 words7 min readBack to topic

Before we start, three plain-word anchors used everywhere below:


True or false — justify

A branch predictor guessing wrong ever corrupts the value in an architectural register.
False — a wrong guess only pollutes microarchitectural state; squashing discards ROB entries before they commit, so the official registers never saw the bad value.
Speculative execution requires out-of-order execution to exist.
False — they are separate ideas. You can speculate past a branch in an in-order pipeline; out-of-order just makes speculation more profitable because more independent work is available to fill the window.
If the branch predictor were 100% accurate, mispredictions would cost zero cycles.
True in the misprediction sense — but you'd still pay the pipeline-fill cost on the very first branch and on cold code, since a predictor with no history still has to make (and verify) a first guess.
Speculatively executed loads can bring data into the cache even when the instruction is later squashed.
True — that is the whole basis of Spectre: the squash undoes the architectural effect but the cache line stays warm, leaking a measurable timing footprint.
A misprediction penalty equals exactly the pipeline depth.
False — it is roughly pipeline depth plus refetch latency (getting the correct instructions back into fetch), so it is a few cycles more than the depth alone.
Speculation is always a net performance win because predictors exceed 95% accuracy on average.
False — average accuracy hides data-dependent branches (parsing, rand()%2) near 50%, where the penalty paid can exceed the cost of simply stalling.
Turning off speculation makes a CPU slower but never changes what results a program computes.
True for correctness of computed values — speculation is designed to be invisible to results; disabling it only removes the timing side effects and the speedup.
The reorder buffer stores instructions in the order they finish executing.
False — the ROB is ordered by program order precisely so it can commit oldest-first; execution finishing order (out-of-order) is decoupled from commit order.

Spot the error

"Once an instruction executes, its result is written to the register, so we just overwrite it on a misprediction."
The error is "written to the register": speculative results go to the ROB, not architectural registers, so there is nothing to overwrite — the wrong entry is simply dropped without ever touching the official register.
"A misprediction flushes the branch instruction itself along with everything behind it."
The branch instruction is correct and is what resolved the mispredict; only the wrong-path instructions after it are squashed, and the branch commits normally.
"Because speculation is invisible to software, Spectre is a software bug, not a hardware issue."
The leak comes from hardware leaving cache traces during speculation; the vulnerable behaviour is in the microarchitecture, so it is fundamentally a hardware design issue that software mitigations only work around.
"A 2-bit saturating counter changes its prediction the first time it's wrong."
A saturating counter has hysteresis — one wrong outcome moves it only one step (e.g. strongly-taken to weakly-taken), so it takes two consecutive misses to flip the actual prediction, which is exactly why loop exits cost only one miss.
"Register renaming is what lets the CPU undo a misprediction."
Renaming exists to remove false data dependencies so instructions can run out-of-order; the undo mechanism is the ROB discarding uncommitted entries — related machinery, but a different job.
"Committing an instruction means it starts executing."
Backwards — commit is the last step (result already computed, moving it to architectural state); execution happened earlier and its result was waiting in the ROB.
"Speculation only ever runs down one predicted path, so it can't be attacked."
Even a single mispredicted path can transiently touch secret memory and leave a cache imprint before the squash, which is precisely the attack surface Spectre exploits.

Why questions

Why does the CPU checkpoint state before speculating instead of after?
Because if the guess is wrong it must restore the exact pre-branch situation; a checkpoint taken after speculation would already be contaminated by wrong-path work.
Why is a deep pipeline both the reason for speculation and the reason mispredictions hurt?
A deep pipeline creates long branch-resolution delays (the bubble speculation fills), but it also means many in-flight wrong-path instructions to flush on a miss — the same depth is both the motivation and the penalty.
Why do some real CPUs deliberately stall rather than speculate on certain branches?
A confidence estimator can detect a near-50/50 branch, where speculating loses more on average than the short wait for resolution — so stalling is the cheaper bet.
Why does sorting input data sometimes speed up branchy code dramatically?
Sorting groups like outcomes together, turning an unpredictable data-dependent branch into long runs of same-direction outcomes the predictor learns easily — cutting the misprediction rate.
Why does branchless code (cmov, bitwise tricks) help unpredictable branches?
It removes the branch entirely, so there is nothing to mispredict and no penalty — trading a possible flush for a guaranteed small computation.
Why must speculative loads NOT update architectural memory immediately?
A wrong-path store could otherwise write bytes that the correct program never intended, corrupting memory that software can see — so stores are held until commit.
Why can a superscalar, out-of-order core extract more benefit from speculation than a simple scalar one?
With multiple issue slots and ILP to exploit, a wider superscalar core has far more independent work ready to fill the speculated window, so each correct guess pays off more.
Why is the branch predictor considered part of the attack surface for Spectre?
An attacker can train it by feeding it patterns, forcing a chosen misprediction that steers the victim down a wrong path touching secret data — the predictor becomes a lever the attacker controls.

Edge cases

What happens to speculation when the very first iteration of a loop runs and the predictor has no history?
It falls back to a default (e.g. weakly-taken) initial state, so early branches may mispredict until the counter warms up — a "cold predictor" cost distinct from steady-state accuracy.
On the last iteration of a for (i=0; i<100; i++) loop, why is a misprediction essentially unavoidable?
The predictor has learned "taken" from 99 taken outcomes, so it confidently predicts taken again exactly when the loop must exit (not taken) — one guaranteed miss at the boundary.
If a mispredicted path would divide by zero or fault, does the program crash?
No — faults on speculative instructions are held pending in the ROB and only raised if that instruction reaches commit; a squashed faulting instruction is discarded silently.
What is the correctness effect of a branch that is always taken every single time it runs?
After warm-up the predictor is right 100% of the time, so it contributes essentially zero misprediction penalty — the ideal case for speculation.
If speculation is fully disabled (e.g. an lfence before every branch), what is the worst-case behaviour?
Every branch stalls until resolved, so throughput drops toward that of a non-speculative machine — correct results, but the pipeline sits idle through each branch delay.
For a purely 50/50 random branch, what does the misprediction rate converge to regardless of predictor sophistication?
About 50% — no predictor can beat coin-flip data, so extra history bits or perceptrons cannot help, and this is the case where stalling may win.
What happens when two nested speculations are in flight and the outer branch mispredicts?
Everything after the outer branch — including the inner speculation and its ROB entries — is squashed together, since the entire path was on wrong ground.
Recall One-line summary of every trap here

Speculation touches hidden state instantly but the official state only at commit — every trap is some place where people confuse "the hardware ran it" with "the program saw it".