Superscalar execution: Visual walkthrough — Hardware deep dive

Before we begin, three words in plain English (we will earn every symbol as we go):

Step 1 — Picture the highway: what "superscalar" even looks like

WHAT. A plain (scalar) processor is a one-lane road: one instruction moves forward per tick. A superscalar processor is a multi-lane highway: several instructions move forward in the same tick.

WHY. Everything on this page is about counting cars crossing a finish line each second. So first we need the picture of the road. The number of lanes has a name.

PICTURE. Below: four lanes, four cars moving together. The dashed finish line on the right is "retire" — where an instruction officially counts as done. Count the cars crossing per tick and you have IPC.

Right now, with no obstacles, all 4 lanes are full, so IPC = 4. That is our first ceiling, and the simplest one:

See Register Renaming and Tomasulo's Algorithm for the machinery that keeps those lanes fed.

Step 2 — Cars need the right kind of exit: functional units

WHAT. Not every instruction is the same. Some are integer adds, some are memory loads, some are floating-point. Each kind must use a matching machine at the far end, called a functional unit.

WHY. Here's the catch that creates the second ceiling. If a lot of your cars all want the same exit, and there are only two of that exit, cars pile up — even if other exits sit idle.

PICTURE. Below, three exit types (integer ×3, memory ×2, FP ×2). All the red memory-cars want the two memory exits. Extra memory cars have to wait, so the highway backs up even though the FP exits are empty.

To turn this picture into a number we need one more word: how often each exit is wanted.

Step 3 — How often is each exit wanted? The instruction mix $p_{i}$

WHAT. Over a long run of a program, a fixed fraction of instructions is of each type.

WHY. If I want to push $X$ instructions through per cycle, then $p_{i} \cdot X$ of them will demand unit type $i$ each cycle. But type $i$ can only serve $F_{i}$ per cycle. So I must never ask for more than it can give: $p_{i} \cdot X \leq F_{i} ⟹ X \leq \frac{F _{i}}{p _{i}}$

Let's read that final ratio term by term:

$instr/cycle I want X \leq \frac{F _{i} units of type i}{fraction wanting them p _{i}}$

PICTURE. A bar for each type. The bar height is that type's ceiling $F_{i} / p_{i}$ . The shortest bar is the true wall — the exit that jams first.

Step 4 — The degenerate case: what if an exit type is never used?

WHAT. Suppose a program has zero floating-point instructions: $p_{FP} = 0$ .

WHY. The formula has $p_{i}$ in a denominator, and dividing by zero is undefined. Do FPUs suddenly limit us to infinity? No — the physics says the opposite: an exit nobody wants can never be a bottleneck.

PICTURE. The FP bar's height is $F_{FP} /0 = + \infty$ — it shoots off the top of the chart. An infinitely tall bar can never be the shortest bar, so $min_{i}$ simply ignores it. That is the correct behaviour.

Step 5 — The hidden rope: dependency chains

WHAT. Some instructions must wait for an earlier one's result. MUL that reads R1 cannot start until the ADD that produced R1 has finished. That is a RAW (Read-After-Write) dependency — a rope tying two cars together.

WHY. A chain of $D$ roped cars, each taking $L$ cycles, needs $D \cdot L$ cycles to clear — and during that time those cars are stuck in single file. To keep the highway busy the CPU must look ahead, find independent cars, and run them while the roped ones wait. How far ahead it can look is the size of its waiting room.

Each cycle the machine pulls up to $W$ new instructions into a ROB of size $ROB$ . The pool refreshes fully every $ROB / W$ ... but the useful bound the parent gives is:

$IPC \leq \frac{W lanes \cdot ROB waiting room}{rope length D \cdot cycles each L}$

Read it as a tug-of-war: a big waiting room ( $ROB$ ) and wide fetch ( $W$ ) push the ceiling up (top of fraction); long ropes ( $D$ ) and slow instructions ( $L$ ) drag it down (bottom of fraction).

PICTURE. A single-file rope of $D$ roped cars on the left (they crawl), and a big ROB pool on the right full of unroped cars the CPU runs meanwhile to keep IPC high.

Step 6 — Put the three walls together: why $min$ ?

WHAT. We now have three separate ceilings. The real IPC is the smallest of the three.

WHY. Think of the machine as a pipe made of three sections of different widths: the front door ( $W$ ), the exits ( $min_{i} F_{i} / p_{i}$ ), and the rope-limited window ( $W \cdot ROB / (D \cdot L)$ ). Water flows only as fast as the narrowest section allows. Widening any other section does nothing. "Narrowest wins" is exactly what $min$ means.

PICTURE. Three walls of different heights. A horizontal red line sits at the height of the shortest wall — that line is your actual IPC. Everything above it is wasted capacity.

Step 7 — Two full walkthroughs, edge behaviour included

The one-picture summary

Below: the whole derivation on a single canvas. Three walls (front door, exits, rope-window); the red finish-line sits at the shortest; that height is IPC. Everything above the line is idle silicon.

Recall Feynman retelling — say it back in plain words

Imagine a multi-lane highway inside the chip. The number of lanes is $W$ — that's the most cars you can ever start each tick. At the far end are exits of different kinds (integer, memory, FP); if too many cars want the same few exits, they jam — that jam limit is $F_{i} / p_{i}$ , and the busiest exit ( $min_{i}$ ) decides. And some cars are roped together and can only go single file; the CPU hides this by keeping a big waiting room (the ROB) full of unroped cars to run in the meantime — that gives $W \cdot ROB / (D \cdot L)$ . Your real speed is set by the narrowest of these three, which is exactly why the formula is a $min$ of three things. If an exit is never used ( $p_{i} = 0$ ), its limit is infinite, so it's simply ignored.

Recall Quick self-test

Why is the master formula a $min$ and not a sum or product? ::: A chain is only as strong as its weakest link — flow through a pipe is set by its narrowest section, so the tightest single ceiling decides IPC. If $p_{FP} = 0$ , what is that exit's ceiling and does it bind? ::: $F_{FP} /0 = \infty$ ; an infinite ceiling can never be the smallest, so it never binds. Machine is fetch-bound at IPC = $W$ . Does adding load/store units help? ::: No — Ceiling #1 is the wall; you must increase $W$ instead. What single ratio is Ceiling #2, and which type sets it? ::: $min_{i} F_{i} / p_{i}$ ; the type with the smallest units-to-demand ratio (the busiest, least-served exit).