Superscalar — multiple execution units

WHAT is it?

WHY does it exist? (first-principles)

A single-issue 5-stage pipeline can finish at most one instruction per cycle. But real code contains instructions that do not depend on each other. Example:

a = b + c     // uses b, c
d = e + f     // uses e, f  — independent of line 1!

If two adders sit idle, we are wasting silicon. The whole motivation is: transistors are cheap, time is precious — replicate functional units and run independent work simultaneously.

HOW: deriving the performance model from scratch

Start from the universal CPU performance equation. We derive it, not quote it.

Time to run a program = (number of instructions) × (time per instruction).

$T = N_{\text{inst}} \times \text{CPI} \times T_{\text{cycle}}$

where $N_{\text{inst}}$ = dynamic instruction count, $\text{CPI}$ = average Cycles Per Instruction, $T_{\text{cycle}}$ = clock period. Now invert CPI:

$\text{IPC} = \frac{1}{\text{CPI}} \quad\Longrightarrow\quad T = \frac{N_{\text{inst}}\, T_{\text{cycle}}}{\text{IPC}}$

Why this matters: a superscalar machine attacks the IPC term. A scalar pipeline caps IPC at 1; an N-wide superscalar raises the ceiling to N. So:

Why we rarely hit IPC = N (Amdahl-flavoured reasoning)

Suppose fraction $p$ of instructions can be parallelized perfectly across N units, and fraction $(1-p)$ must run serially (chained dependencies). Then effective IPC is bounded:

$\text{IPC}_{\text{eff}} = \frac{1}{(1-p) + \dfrac{p}{N}}$

The three obstacles (and how superscalars fight them)

Weapons:

• Register renaming removes false dependencies (WAR, WAW) by mapping the same architectural register name to different physical registers.
• Out-of-order execution + Reorder Buffer (ROB) lets ready instructions overtake stalled ones, then retires them in order for correctness.
• Branch prediction + speculation keeps the units fed past branches.
• Multiple ports on register file / caches dissolve structural hazards.

Worked Example 1 — counting cycles, in-order, 2-wide

Code (each op takes 1 cycle, latency 1):

I1: r1 = r2 + r3
I2: r4 = r5 + r6     (independent of I1)
I3: r7 = r1 + r4     (needs r1 from I1, r4 from I2)
I4: r8 = r9 + r10    (independent)

2-wide, in-order, two integer ALUs.

Total = 2 cycles for 4 instructions → IPC = 2 (peak!). A scalar machine would take 4 cycles (IPC = 1). Speedup = 2×.

Worked Example 2 — a dependency chain destroys ILP

I1: r1 = r2 + r3
I2: r4 = r1 + r5     (needs r1)
I3: r6 = r4 + r7     (needs r4)
I4: r8 = r6 + r9     (needs r6)

Every instruction depends on the previous one.

4 cycles, IPC = 1 — no benefit from the second unit. This is $p=0$ in our formula. Steel-man lesson: width helps only if independent work exists.

Worked Example 3 — false dependency fixed by renaming

I1: r1 = r2 + r3
I2: r1 = r4 + r5     (WAW: writes r1 again — but value independent!)

In-order naive view says "both write r1, serialize." But the values are independent. Rename: I2 writes physical p7 instead of reusing r1's physical reg. Now they issue together, IPC = 2.

Why this step? Renaming proves the conflict was a naming artifact, not a real data flow.

Flashcards

Connections

• Pipelining — superscalar builds on a pipelined datapath.
• Instruction-Level Parallelism (ILP) — the resource superscalar exploits.
• Out-of-Order Execution and Register Renaming — enable high real IPC.
• Reorder Buffer (ROB) — guarantees in-order retirement.
• Branch Prediction — keeps wide units fed past control hazards.
• Amdahl's Law — same math governs IPC_eff.
• VLIW — compiler-scheduled alternative to hardware superscalar.
• Data Hazards (RAW WAR WAW) — what limits issue width.

Concept Map

Hinglish (regional understanding)