Superscalar execution: Question bank — Hardware deep dive

This is a question bank for Superscalar execution. Every item is a one-line reveal: read the prompt, say your answer out loud with a reason, then check. The answer side always gives the reasoning — a bare "yes/no" is worthless for building intuition.

Prerequisites worth re-reading if a group hurts: Register Renaming, Tomasulo's Algorithm, Reorder Buffer, Branch Prediction, VLIW Architecture, Cache Hierarchy, Power-Performance Tradeoffs.

First: the five symbols this whole page leans on

Before you attempt the traps, you must be able to read every letter in the IPC limit formulas. None of them are fancy — each is just a count or a fraction. Build the picture once here so no later reveal uses an undefined symbol.

Definition The five quantities

Issue width $W$ — how many instructions the front end can fetch/decode/dispatch in one clock cycle. "Dispatch" is the moment a renamed instruction is handed to the out-of-order back end; the narrowest of the fetch/decode/dispatch stages sets $W$ . Picture $W$ parallel lanes on a highway; $W = 4$ means four cars enter per tick.
Functional-unit count $F_{i}$ — how many copies of execution unit type $i$ exist (e.g. $F_{ALU} = 3$ means three integer adders). These are the workers that actually do the arithmetic.
Instruction-mix fraction $p_{i}$ — of all the instructions your program runs, the fraction that need unit type $i$ . If $30%$ of instructions are loads/stores then $p_{mem} = 0.3$ . Because every instruction needs exactly one unit type, these fractions form a complete partition: $\sum_{i} p_{i} = 1$ .
Reorder-buffer size $ROB$ — the number of entries in the Reorder Buffer, the circular queue that tracks in-flight instructions in program order. So " $ROB$ " in every formula below literally means how many instructions the machine can have in flight at once. See Reorder Buffer.
Dependency length $D$ and latency $L$ — $D$ is the average length of a chain "this instruction needs the result of that one, which needs the result of another..."; $L$ is how many cycles the average instruction takes to produce its result once it starts.

That is five named things — one width $W$ , one unit-count $F_{i}$ , one fraction $p_{i}$ , one buffer size $ROB$ , and the dependency pair $(D, L)$ — and nothing on this page uses any symbol outside that list.

Why the functional-unit limit is $F_{i} / p_{i}$

Here is the reasoning, not just the formula. Suppose you want to retire $x$ instructions every cycle. A fraction $p_{i}$ of them need unit type $i$ , so of those $x$ instructions, $p_{i} \cdot x$ demand a type- $i$ unit this same cycle. But you only own $F_{i}$ such units, so you can never serve more than $F_{i}$ of them per cycle:

$p_{i} \cdot x \leq F_{i} ⟹ x \leq \frac{F _{i}}{p _{i}}$

That is why dividing units by fraction gives an IPC ceiling: it's "how many total instructions can flow before the scarce unit is saturated." Because $\sum_{i} p_{i} = 1$ , no unit type can be ignored — the fractions genuinely add up to the whole workload, so the smallest $F_{i} / p_{i}$ is the true bottleneck. Look at the leftmost panel (Panel 1, titled "Unit limit F/p") of the figure — the teal memory-demand line $p_{mem} \cdot x$ rises until it crosses the orange worker-count line at $x = F / p$ , the point where that unit saturates.

Why the dependency limit is $\frac{W \cdot ROB}{D \cdot L}$

Build this in three "why" steps.

Why $ROB$ on top? The reorder buffer holds at most $ROB$ instructions in flight at once, so that is the size of the window the hardware can search for independent work. A bigger window exposes more parallelism.

Why divide by $D \cdot L$ ? A single dependency chain of length $D$ cannot go faster than one link at a time, and each link takes $L$ cycles to produce its result — so that whole chain needs $D \cdot L$ cycles to finish. Meanwhile the window of $ROB$ instructions holds about $ROB / D$ independent chains side by side. Those independent chains interleave: while chain A waits $L$ cycles for its link to finish, the lanes run chains B, C, D, ... So over the $D \cdot L$ cycles the window takes to drain, it completes roughly $ROB$ instructions — a rate of $\frac{ROB}{D \cdot L}$ instructions per cycle.

Why multiply by $W$ ? That rate is what the back end can sustain, but instructions still have to enter through the front end. Each cycle the front end can inject at most $W$ new instructions to refill the window as it drains. If the back end could in principle drain faster than $W$ per cycle, the front end throttles it — so $W$ multiplies (caps) the refill of the interleaving window. Hence $\frac{W \cdot ROB}{D \cdot L}$ .

Common mistake What "latency

$L$ " secretly bundles — and its edge cases The steady-state formula hides all stall sources inside the average latency $L$ and dependency length $D$ . Two big ones deserve their own accounting:

Load misses inflate $L$ : a normal load has $L \approx 3$ – $4$ ; an L2 or main-memory miss can push that single instruction's effective latency to $20$ – $300$ cycles (see Cache Hierarchy). It is not the same $L$ as an add — treat a miss as a local spike $L_{miss}$ , and the machine only hides it while independent work in the ROB window lasts.
Branch mispredicts don't fit $L$ at all — they feed $D$ and waste the whole window. A mispredict flushes the speculatively fetched instructions and restarts fetch, effectively resetting the interleaving window; model it as a per-mispredict penalty $P_{mp}$ cycles times the mispredict rate, subtracted from useful throughput (see Branch Prediction).

The lesson: whenever someone says "just raise $L$ ", ask which stall — a memory spike behaves differently from a control-flow flush, and lumping both under one average $L$ hides exactly the edge cases the exercises below probe.

The middle panel (Panel 2, titled "Dependency limit") shows a full window draining one dependent chain of length $D$ : each teal/plum block executes $L$ cycles after the one it depends on, and the orange arrows are the "needs the result of" links — the longer the chain, the more the other lanes sit idle.

True or false — justify

A pipelined processor and a superscalar processor are the same thing

False. A plain pipeline has one instruction in each stage — it overlaps instructions but still starts and finishes at most one per cycle; a superscalar has multiple parallel lanes so it can start and retire several per cycle.

A 4-wide superscalar always achieves 4 IPC

False. 4 is the ceiling (the fetch/decode/dispatch width

W

); the real IPC is the minimum of that width, the functional-unit limits

F_{i} / p_{i}

, and the dependency limit — any one can drop it well below 4.

Out-of-order execution means instructions also retire out of order

False. Instructions execute out of order but the Reorder Buffer forces them to commit in program order, which is what preserves precise exceptions.

False. RAW is a true data dependency — the reader genuinely needs the value being written — so no renaming can remove it; renaming only kills the false WAW and WAR hazards.

Renaming removes WAW hazards by making the two writes happen at the same time

False. It removes them by sending the two writes to different physical registers, so their order no longer matters — nothing is forced to be simultaneous.

Adding more functional units always raises IPC

False. Extra units raise a single

F_{i} / p_{i}

term, so they only help if that term was the smallest one; if width

W

or the dependency limit was the bottleneck, the new units sit idle.

A larger reorder buffer can only help performance, never hurt it

Mostly false in practice. A bigger ROB raises the dependency ceiling

W \cdot ROB / (D \cdot L)

, but it costs area, ports, and power and lengthens the search logic — a classic power–performance tradeoff where more silicon buys diminishing IPC.

If every instruction depends on the previous one, an 8-wide superscalar behaves like a 1-wide one

True. A fully serial chain makes

D

equal to the program length, so the dependency limit

W \cdot ROB / (D \cdot L)

collapses toward 1 no matter how wide the machine is.

VLIW and superscalar solve instruction-level parallelism the same way

False. VLIW Architecture lets the compiler pack independent ops into one wide word (static scheduling); superscalar hardware dynamically discovers parallelism at run time via renaming and reservation stations.

A superscalar can hide a cache miss as long as its ROB has free entries

Partly true. It keeps issuing independent work while the miss resolves, but the missing load sits at the ROB head blocking retirement; once the ROB fills, dispatch stalls regardless of width — a miss is a local spike in

L

that eventually drains the cache-fed window.

Spot the error

"We fetch 4 per cycle, so we retire 4 per cycle."

Wrong link: fetch width

W

bounds the input rate, but retirement is bounded by whichever instruction is stuck (a slow load, a dependency, or a full unit); average retire rate ≤ fetch rate, often strictly less.

"Since renaming gives each write its own physical register, we never run out of registers."

Wrong: the physical register file is finite; when it empties, rename must stall until an in-flight instruction retires and frees its old physical register — the Register Renaming pool is a resource like any other.

"The instruction mix is 60% memory and we have 2 load/store units, so memory throughput is

2 \times 0.6 = 1.2

IPC."

Wrong formula: the memory limit is

F_{i} / p_{i} = 2/0.6 \approx 3.33

IPC (units divided by the fraction needing them), not units times the fraction.

"Branches are just another instruction type; they don't affect superscalar width utilization."

Wrong: a mispredicted branch flushes all the speculatively fetched instructions behind it, so Branch Prediction accuracy directly caps how much of the width

W

is actually useful.

"A micro-op cache raises the fetch width, so it raises the theoretical IPC ceiling."

Two confusions. First, the micro-op cache sits after decode — it stores already-decoded micro-ops so the pipeline can skip fetch-and-decode on a hit, easing the decode bottleneck (x86 variable-length boundary-finding) and saving power, not widening raw fetch. Second, the ceiling

W

is set by the narrowest of fetch/decode/dispatch; dispatch (rename + hand-off to reservation stations) is usually the true limiter, and the micro-op cache does not widen it.

"Two instructions writing the same architectural register can never run in parallel."

Wrong: after renaming they write different physical registers, so they can run in parallel; only the mapping in the RAT must record which one is 'latest'.

"With a 128-entry ROB and average latency 2, the dependency limit is 128/2 = 64 IPC."

Missing terms: the limit is

\frac{W \cdot ROB}{D \cdot L}

; you dropped the width

W

in the numerator and the dependency length

D

in the denominator.

Why questions

Why must retirement be in program order even though execution is not?

So that on an exception every earlier instruction has committed and no later one has left a visible effect — the precise exception guarantee the OS relies on.

Why does variable-length x86 encoding make wide decode especially hard?

You can't find the start of instruction 2 until you know the length of instruction 1, so parallel boundary-finding needs pre-decode bits or complex speculative decoders.

Why does Tomasulo's Algorithm use reservation stations instead of decoding directly to execution?

Reservation stations hold instructions and snoop the result bus for their operands, letting an instruction wait for its data without blocking others behind it in program order.

Why is a wide fetch useless without a good branch predictor?

Real code branches roughly every 5–7 instructions, so without predicting past branches you'd rarely have enough consecutive instructions to fill a wide fetch window of width

W

Why does the IPC formula take a min over all limits instead of an average?

Throughput is set by the tightest constraint — the slowest stage or scarcest resource — so the worst of

W

min_{i} F_{i} / p_{i}

and the dependency limit governs, exactly like the narrowest section of a pipe sets its flow.

Why can renaming eliminate WAR (anti-dependency) hazards?

The later write is redirected to a fresh physical register, so it can't overwrite the value the earlier read still needs — the two now touch different storage.

Why is it dangerous to hide both cache misses and branch mispredicts inside one "average latency

L

A load miss inflates

L

locally but stays hideable while independent work lasts; a mispredict flushes the whole window and resets fetch — modeling both as the same

L

erases that different behaviour and the edge cases it creates.

Edge cases

What is the IPC of a superscalar running a single long chain of dependent adds (D = N)?

Roughly 1 — with

D

huge the dependency limit

W \cdot ROB / (D \cdot L)

falls near 1, because each add waits for the previous result and the extra lanes are wasted.

What happens to a 4-wide machine if all four instructions in a bundle need the one FP-divide unit?

A structural hazard:

F_{i} = 1

so only one issues that cycle while the other three wait in reservation stations — this bundle achieves 1 IPC on that unit despite the width.

What if the instruction mix has

p_{i} = 0

for some unit type?

That unit imposes no limit —

F_{i} / p_{i}

with

p_{i} = 0

is "infinite headroom", so you drop that term from the min rather than computing it; the remaining fractions still sum to 1.

What is the maximum in-flight instruction count, and what caps it?

It equals the ROB size; once the reorder buffer is full, dispatch stalls even if functional units are idle, because there's nowhere to track a new instruction's ordering.

What limits a superscalar on straight-line, fully-independent code with a huge ROB?

The fetch/decode/dispatch width

W

alone — no dependencies (

D \approx 1

) and no unit shortages means the min pins IPC to the front-end width, the best case of the formula.

What happens on a branch mispredict deep in a full ROB?

All instructions after the branch are squashed and the RAT is rolled back to the branch's mapping; the deeper the speculation, the more work is thrown away — a per-mispredict penalty

P_{mp}

subtracted from useful throughput, and a key cost of aggressive Branch Prediction.

Recall Two-line self-test before you leave

Fetch vs retire — which bounds which, and picture why? ::: Fetch width $W$ is the entrance to the highway; retire is the exit, throttled by the slowest stuck car (a cache-missed load, a dependency, a saturated unit). Sustained IPC = exit rate ≤ entrance rate $W$ , and usually strictly less because some lane is always waiting. The three terms inside the IPC min, and the mental image for each? ::: Width $W$ = highway lanes; $min_{i} F_{i} / p_{i}$ = the scarcest toll booth (workers ÷ demand-fraction, and the fractions sum to 1); $\frac{W \cdot ROB}{D \cdot L}$ = how much independent work the ROB window can expose against chains of length $D$ . The narrowest of the three sets the flow.