Superscalar — multiple execution units: Question bank

Before the traps, we must earn every symbol this page uses. Do not skip this glossary — the questions below lean on all four quantities.

Definition Every symbol on this page, in plain words

Cycle — one tick of the CPU's clock, the smallest unit of "time" the hardware counts in.
$T_{cycle}$ (clock period) — how many seconds one cycle lasts. A 3 GHz chip has $T_{cycle} \approx 0.33$ nanoseconds. Making it smaller means a faster clock.
IPC (Instructions Per Cycle) — on average, how many instructions the CPU finishes in one cycle. A plain pipeline manages at most 1; a superscalar aims for more.
CPI (Cycles Per Instruction) — the flip side: how many cycles each instruction costs on average. They are reciprocals: $IPC = 1/ CPI$ . "IPC = 2" and "CPI = 0.5" say the same thing.
$N$ (issue width) — how many instructions the hardware can dispatch to execution units in one cycle. Picture $N$ side-by-side lanes; $N$ is the hard ceiling on IPC, because you can't finish more than you can start.
$p$ (parallelizable fraction) — the fraction of instructions in the program that are independent enough to run alongside others. $p = 1$ means everything can overlap; $p = 0$ means every instruction waits on the one before it (a serial chain). The leftover $(1 - p)$ is the part that must run one-at-a-time.

Now the three anchor pictures the trap questions refer to — look at the figure before reading on:

Figure — Superscalar — multiple execution units

Width = several execution units side by side (more checkout lanes). Raises the IPC ceiling to $N$ .
Depth = more pipeline stages (thinner slices, faster clock) — shrinks $T_{cycle}$ . See Pipelining.
Order = execute whenever ready, but commit in program order — see Out-of-Order Execution and Reorder Buffer (ROB).

True or false — justify

A 4-wide superscalar always runs code 4× faster than a scalar CPU.

False —

N = 4

is only the ceiling on IPC; real code hits data and control hazards, so a 4-wide out-of-order core typically sustains IPC ≈ 1.5–2.5, not 4. See Amdahl's Law applied to ILP.

Superscalar and superpipelining are the same idea.

False — superscalar adds width (more execution units, lowers CPI/raises IPC); superpipelining adds depth (more stages, raises clock frequency, shrinks

T_{cycle}

). They are orthogonal and often combined.

An in-order superscalar can never beat IPC = 1.

False — if two adjacent instructions are independent it issues both in one cycle, giving IPC up to

N

; it only stalls when the next instruction in program order isn't ready.

Out-of-order execution means the program's results appear in the wrong order.

False — instructions execute out of order, but the Reorder Buffer (ROB) retires (commits) them in program order, so architectural state and exceptions stay exactly as the program specified.

False — renaming only kills false dependencies (WAR, WAW) that come from reusing register names; a RAW dependency is a real data flow (the value must be produced first) and cannot be renamed away. See Data Hazards (RAW WAR WAW) and Register Renaming.

Widening the issue width helps even a purely serial dependency chain.

False — a chain has

p = 0

IPC_{eff} = \frac{1}{( 1 - p ) + p / N}

, giving IPC = 1 for any

N

; extra units sit idle because there is no independent work to fill them.

IPC and CPI describe different things.

False — they are reciprocals:

IPC = 1/ CPI

. Saying "IPC rose from 1 to 2" is identical to "CPI fell from 1 to 0.5".

A superscalar processor needs branch prediction to be useful.

Mostly true — without prediction the machine stalls at every branch and can't keep multiple units fed past it; Branch Prediction and speculation supply a stream of instructions so the wide back-end doesn't starve. See Instruction-Level Parallelism (ILP).

Making each instruction's latency shorter is the main source of superscalar speedup.

False — each instruction still takes the same latency; the win is throughput, more instructions retiring per unit time because several flow in parallel.

Spot the error

"Our 3-wide CPU decodes 3 instructions, so it must complete 3 every cycle."

Error: decoding 3 ≠ completing 3. Completion also needs those 3 to be independent and to find free execution units; dependencies or a structural conflict can drop actual IPC below 3.

"To fix the WAW conflict in r1=r2+r3; r1=r4+r5, we serialize the two writes."

Error: that's a false dependency. The two values are independent; Register Renaming maps the second write to a different physical register so both issue together — no serialization needed.

"IPC = N because we have N execution units."

Error: N units set the maximum IPC. Real IPC is

\frac{1}{( 1 - p ) + p / N}

; only perfectly parallel code (

p = 1

) reaches N.

"We added a second ALU, so all RAW hazards disappear."

Error: a second ALU dissolves structural hazards (two ops needing one unit), not data hazards. A RAW hazard is about a value not yet produced — more hardware can't produce it earlier.

"Out-of-order execution makes register renaming unnecessary."

Error: they are complementary. OoO lets ready instructions overtake stalled ones, but without renaming the false WAR/WAW dependencies would force ordering and block the overtaking. See Out-of-Order Execution.

"A VLIW machine and a superscalar machine are the same because both issue multiple ops per cycle."

Error: in VLIW the compiler bundles independent ops into one long word at compile time; a superscalar's hardware finds independence dynamically at run time. Same goal, opposite mechanism.

"IPC_eff = 1/((1-p) + p/N), so at p = 0.5, N = 4 we get IPC = 4."

Error: substitute correctly —

\frac{1}{0.5 + 0.5/4} = \frac{1}{0.625} = 1.6

, far below 4. Half-serial code caps hard.

Why questions

Why does throughput, not latency, define superscalar's benefit?

Because each instruction's own execution time is unchanged; the gain comes from overlapping independent instructions across parallel units, so more retire per unit time. Same trip length, more cars on the road.

Why must an out-of-order core still retire instructions in program order?

To preserve correct architectural state and precise exceptions — if a later instruction committed first and an earlier one faulted, the visible machine state would be inconsistent. The Reorder Buffer (ROB) enforces in-order commit.

Why do dependency chains cap IPC at 1 regardless of width?

Each link needs the previous result before it can even start, so at most one link advances per cycle; the other units have no ready work. This is the

p = 0

case of Amdahl-style reasoning.

Why does register renaming increase available parallelism?

It reveals that WAR/WAW conflicts were naming artifacts, not real data flow; giving each write its own physical register lets independent instructions issue simultaneously instead of waiting on a shared name.

Why does a wider issue width demand more read/write ports on the register file?

Issuing N instructions per cycle can require reading up to

2 N

operands and writing N results at once; too few ports creates a structural hazard that throttles the wide front-end.

Why is branch prediction critical specifically for wide machines?

A wide back-end consumes many instructions per cycle, so a mispredicted or unresolved branch starves multiple units at once — the wider the machine, the more work lost per stall. See Branch Prediction.

Why can two independent instructions still fail to issue together on a real superscalar?

Even if data-independent, they may compete for the same execution unit (structural hazard) or exceed the issue width or port budget in that cycle.

Edge cases

If p = 0 (fully serial), what does IPC_eff equal for any N, and why?

IPC_{eff} = \frac{1}{( 1 - 0 ) + 0/ N} = 1

. A serial chain advances one instruction per cycle no matter how many units exist.

If p = 1 (fully parallel), what does IPC_eff equal, and is it achievable in practice?

IPC_{eff} = \frac{1}{0 + 1/ N} = N

. It's the theoretical peak; real code never sustains it because some fraction is always serial or hazard-bound.

For N = 1 (a scalar single-issue pipeline), what does IPC_eff reduce to?

\frac{1}{( 1 - p ) + p /1} = \frac{1}{1} = 1

for any p — with one unit there is no parallelism to exploit, so p becomes irrelevant.

What happens to speedup as N → ∞ for fixed p < 1?

The

p / N

term vanishes, so IPC saturates at

\frac{1}{1 - p}

, a hard ceiling set by the serial fraction — pouring in units gives diminishing returns. This is Amdahl's Law's asymptote.

A block of code has zero independent instructions and infinite issue width — expected IPC?

Still exactly 1; with no independent work, only one instruction per cycle can advance regardless of hardware width.

Can IPC ever exceed the issue width N?

No — N is the number of instructions that can be dispatched per cycle, so it is a strict upper bound; you cannot retire more per cycle than you can issue.

Does a program with IPC already equal to N gain anything from doubling N?

No further steady-state gain — it already saturates the units it uses; extra width only helps if there are more simultaneously-ready independent instructions to fill the new lanes.

Recall One-line self-test

The single sentence that ties the whole page together? Answer ::: Width raises the ceiling on IPC, but only independent work + correct ordering machinery (renaming, ROB, prediction) lets real code approach it — and a serial chain caps IPC at 1 forever.