Superscalar execution: Foundations — Hardware deep dive

Before you can read the parent note Superscalar execution, you must be able to see every symbol it throws at you. Below, each idea is built from nothing: plain words → a picture → why the topic needs it. Read top to bottom; each block leans on the one above.

1. A "clock cycle" — the heartbeat

The picture. Imagine a drummer keeping a steady beat. On every beat, gates open and work moves one step forward. The beat never speeds up mid-song — the whole design is built around fixed-length ticks.

Figure s01 — walkthrough. The pale-yellow square wave is the clock signal; it snaps between "low" and "high". Each dotted blue vertical line marks the rising edge that begins a new tick, and the pink labels ("tick 1", "tick 2", …) count them. Notice the spacing between the blue lines is identical everywhere — that fixed width is the whole point: a cycle is a unit of time you can count.

Why the topic needs it. Everything is counted "per cycle": instructions fetched per cycle, retired per cycle. If you don't picture the tick, phrases like " $4$ instructions per cycle" are just noise.

2. An "instruction" and the "pipeline" — the assembly line

The picture. A single conveyor belt with stations. One car per station. One car rolls off the end every tick.

Figure s02 — walkthrough. Read the grid as columns = clock cycles (c1…c8, bottom axis) and rows = pipeline stages (Fetch, Decode, Exec, Write, left axis). Each coloured box holds an instruction label (I1…I4). Follow instruction I1 on the diagonal staircase: it is Fetched in c1, Decoded in c2, Executed in c3, Written in c4. The staircase shape means four instructions are always inside the pipe at once — but trace the bottom "Write" row: only one box lands there per column (the pink arrow points at it). That is the pipeline's hard limit: overlap in time, yet only one finish per tick.

Why the topic needs it. Superscalar is defined against the plain pipeline. A plain pipeline still finishes only one instruction per tick. The whole point of the parent note is: what if we run several belts side by side?

3. IPC — the score we are chasing

A plain pipeline: $IPC \leq 1$ (at best one per tick).
A $4$ -wide superscalar: $IPC$ can approach $4$ .

Why the topic needs it. The parent's whole "Derivation: Maximum IPC" is about the ceiling on this number. Every limit (units, width, dependencies) is a reason IPC falls below the ideal.

4. The symbol $W$ — issue width

The picture. Four belts side by side instead of one. Four cars enter every tick.

This is our first of several independent ceilings. Keep it in mind — we will collect three of them and combine them at the end.

5. Functional units and the symbols $F_{i}$ , $p_{i}$

The subscript $i$ is just a label ("ALU", "FP", "mem", "branch"). It is not a number.

Figure s03 — walkthrough. Each bar is one unit type; its height is the ceiling

F_{i} / p_{i}

(instructions/cycle), printed on top. Read the ALU bar:

3

units serving a

40%

demand gives

3/0.4 = 7.5

. Now scan for the shortest bar — outlined in pink here, the memory unit at

6.67

. That shortest bar is the traffic jam: no matter how tall the others are, the whole machine can only flow as fast as its most-starved unit type. The pink arrow labels it "bottleneck = smallest F/p".

Now let us make that ceiling formal, matching the width ceiling above.

6. Dependencies — RAW, WAW, WAR (why order matters)

Two instructions can only run at the same time if they don't step on each other. There are three ways they can clash, using a register $R_{k}$ (a named storage slot, like $R 1$ ).

Figure s04 — walkthrough. Three little stacked pairs, each an instruction A above instruction B sharing register

R 1

. Left, blue (RAW): the yellow arrow runs A→B because B literally needs the value A produced — labelled TRUE. Lower-left, pink (WAW): both write

R 1

; they clash only over the name — labelled false. Right, off-white (WAR): A reads

R 1

then B overwrites it — again false. The blue note at the bottom is the punchline: rename the second writer to a fresh name and both false clashes disappear.

Why "true" vs "false" matters. RAW is real data flow — no trick removes it. WAW and WAR exist only because we reused a register name. Give the second writer a fresh name and the clash vanishes. That trick is Register Renaming.

Why the topic needs it. The whole "Rename and Dispatch" stage, and the parent's dependency-chain length $D$ , are about these hazards.

7. The ROB — the instruction "waiting room"

Before we can write the parent's dependency ceiling we need one more symbol: the size of the buffer that holds in-flight instructions.

The picture. A ticket queue: people are served (compute) in any order, but they leave through the door one-by-one in the order they arrived — so from the outside everything looks orderly. $R_{ob}$ is how many tickets the queue can hold.

Why the topic needs it. A bigger ROB = a bigger window to hunt for independent work = higher IPC. That's why $R_{ob}$ sits inside the IPC formula built in the next section.

8. $D$ and $L$ — chain length and latency

Deriving the dependency ceiling — step by step. We now have every symbol to build the parent's third ceiling. It is still ordinary IPC (section 3), but computed under one specific limit: the supply of independent work.

Formula The dependency ceiling — derived

Step 1 — how many slots of work are available at once? The machine fetches up to $W$ instructions per cycle and can hold $R_{ob}$ of them in flight. Over the lifetime of a full window, the total pool of instruction-slots it has staged is $slots available = W \cdot R_{ob} .$ Step 2 — how many slots does one dependency chain tie up? A RAW chain of length $D$ cannot overlap with itself; each of its $D$ instructions takes $L$ cycles to produce its result, so finishing the chain occupies $slots occupied per chain = D \cdot L$ cycle-slots that are forced to run strictly in order. Step 3 — divide available by occupied. The number of chains we can keep flowing per cycle — and hence the IPC ceiling dependencies allow — is the pool divided by the per-chain cost: $IPC_{dependency} \leq \frac{W \cdot R _{ob}}{D \cdot L} .$

Read the shape: a longer chain ( $D$ big) or slower ops ( $L$ big) shrink the ceiling; a bigger ROB ( $R_{ob}$ big) grows it by giving more independent work to overlap. This is the third and last ceiling.

9. Control-flow dependencies — the branch ceiling

There is one more way to stall that isn't RAW/WAW/WAR at all.

Why this is its own ceiling. A misprediction empties the whole pipeline and re-fetches from the correct target, wasting a fixed number of cycles (the misprediction penalty). The more branches, and the worse the prediction, the more often the wide fetch engine runs dry. So even with perfect units and infinite ROB, control flow caps IPC:

10. Combining the ceilings — why minimum?

We collected three independent ceilings, each a true upper bound that must hold on its own:

$IPC \leq W, IPC \leq min_{i} \frac{F _{i}}{p _{i}}, IPC \leq \frac{W \cdot R _{ob}}{D \cdot L} .$

The prerequisite map

Each foundation flows into the IPC score, and the IPC score is what the parent note derives a ceiling for.

Once these symbols are solid, the machinery that implements them lives in: Register Renaming · Tomasulo's Algorithm · Reorder Buffer · Branch Prediction · Cache Hierarchy · VLIW Architecture · Power-Performance Tradeoffs. And the Hindi walkthrough: 5.3.01 Superscalar execution (Hinglish).

Equipment checklist

Read the question, answer aloud, then reveal.

What does one clock cycle represent?

One tick of the CPU's metronome; all work is measured per tick.

How is a plain pipeline different from a superscalar?

A pipeline overlaps stages of instructions in time (1 finish/tick max); a superscalar runs multiple belts so several instructions share the same stage and several finish per tick.

Define IPC in one sentence.

The average number of instructions the CPU finishes per clock cycle.

What is the symbol

W

and what ceiling does it place on IPC?

Issue width — instructions started per cycle; you can never finish more than you start, so

IPC \leq W

What do

F_{i}

and

p_{i}

mean, and what must

\sum_{i} p_{i}

satisfy?

F_{i}

= number of functional units of type

i

;

p_{i}

= fraction of instructions needing that type; the classes are disjoint so

\sum_{i} p_{i} \leq 1

Why is the unit ceiling

F_{i} / p_{i}

a division, and where does it come from?

From

p_{i} x \leq F_{i}

(demand can't exceed supply), giving

x \leq F_{i} / p_{i}

; supply over demand-fraction.

Name the three hazards and which are false.

RAW (true), WAW (false), WAR (false); the false ones vanish with register renaming.

What does the symbol

R_{ob}

stand for?

The size of the reorder buffer — how many instructions can be in flight at once.

What do

D

and

L

stand for?

D

= length of a RAW dependency chain;

L

= cycles for one instruction to produce its result.

In the dependency ceiling, what counts as available slots vs occupied slots?

W \cdot R_{ob}

= staged instruction-slots available;

D \cdot L

= cycle-slots one in-order chain occupies.

Why do the three ceilings combine with a minimum?

Each is an upper bound that must hold at once, so IPC cannot exceed the smallest — the tightest bound wins.

Besides RAW/WAW/WAR, what other dependency caps IPC?

Control-flow (branch) dependencies — mispredictions flush the pipeline and waste staged work.

Latency vs throughput — the difference?

Latency = time for one op to finish; throughput = how many ops start per cycle.

What does the ROB provide and why does its size matter?

In-order commit despite out-of-order execution (precise exceptions); a bigger ROB gives a wider window to find independent work, raising IPC.