Deep pipelining trade-offs: Foundations — Hardware deep dive

0. The picture everything sits on: the assembly line

Before any symbol, look at what a pipeline is.

Each instruction (a car body) moves left to right through fixed stations (stages). At every clock tick, everybody shifts one station to the right at the same instant. The red car is the one we follow; the black cars behind it are already being worked on too. That "everybody at once" is the whole trick — while one instruction is being decoded, another is being fetched.

Everything below is a number that describes this picture.

1. Time per tick: $T_{cycle}$

The picture: in the assembly-line figure, $T_{cycle}$ is the length of the pause between shifts. Make it shorter → cars move more often → more cars finish per second.

Why the topic needs it: the entire game of deep pipelining is shrinking $T_{cycle}$ . Every trade-off is ultimately "did $T_{cycle}$ get smaller, and at what cost?"

Because all stations shift together, they all get the same amount of time. So $T_{cycle}$ can be no shorter than the slowest station needs — a chain is as slow as its slowest link.

2. Ticks per second: $f$ and why $f = 1/ T_{cycle}$

Why this exact formula? If each tick takes $T_{cycle}$ seconds, then in one whole second you fit $f = \frac{1}{T _{cycle}} .$ This is just "how many 2-ns gaps fit in 1 second?" — it is not physics, it is counting. We use the reciprocal (one-over) because "time per event" and "events per time" are always reciprocals, the way "seconds per lap" and "laps per second" are.

Why the topic needs it: designers advertise frequency, but frequency is only a disguised statement about cycle time. When the parent says "more stages → higher $f$ ", it literally means "thinner stages → smaller $T_{cycle}$ ".

3. Work in one stage: $τ$ (the "combinational delay")

The picture: inside each station, signals ripple through logic gates like dominoes falling. $τ$ is how long the last domino takes to fall.

Why a Greek letter? Just tradition — engineers reserve $τ$ for "a small delay". No hidden meaning; treat it as "the letter for one stage's think-time".

Why the topic needs it: splitting one big station into two means each half has a smaller $τ$ . Smaller $τ$ → shorter cycle → higher $f$ . This is the entire motivation for going deeper.

4. The glue between stations: pipeline registers and $t_{reg}$

Here is the hidden cost the assembly-line picture didn't show. Between two stations there is a holding shelf that grabs a stage's result and holds it steady so the next stage can read it cleanly.

That shelf is not instant. It costs time made of three pieces:

Why the topic needs it — the crucial one: $t_{reg}$ is a fixed cost that does not shrink when you add stages. $τ$ shrinks as you split; $t_{reg}$ does not. So the real cycle time is $T_{cycle} = τ + t_{reg},$ and no matter how thin you slice, you can never get below $t_{reg}$ . This one fact is the wall the whole parent note circles around.

5. Counting stations: $k$ and the total work $T_{0}$

Before we can say what happens as $k$ grows, we need a name for "the whole job" that the stages together perform.

The picture: $T_{0}$ is the full length of the domino chain from §3 before you cut it into stations. Cut it into $k$ equal pieces and each piece is $T_{0} / k$ long.

Why the topic needs it: with $T_{0}$ named, we can write cycle time as a function of depth $k$ . Read $T_{cycle} (k)$ as "the cycle time you get when you choose $k$ stages" — the parentheses just mean " $T_{cycle}$ depends on $k$ ", the way " $price (weight)$ " means price depends on weight.

$T_{cycle} (k) = one stage’s τ \frac{T _{0}}{k} + t_{reg} .$

As $k$ grows the first term melts toward zero and $T_{cycle} (k) \to t_{reg}$ — the register floor. That limit is why "deeper" stops helping.

6. Work per tick, and total throughput: CPI and $R$

Frequency alone lies, because not every tick does useful work. We need a second number.

The picture: picture the assembly line hiccupping — sometimes a station has to freeze (a bubble, an empty slot) while waiting. Each frozen tick is a wasted cycle spread across the instructions, raising CPI above the ideal $1$ .

Now we can name the thing we actually care about — instructions finished per second.

Why the topic needs it: deep pipelines raise $f$ but also raise CPI (more hiccups). If both rise by the same factor, $R$ is unchanged — you gain nothing, exactly the Pentium 4 story. $R$ is the accountant that catches the hidden losses. The causes of those hiccups have their own note: Pipeline Hazards.

7. Why hiccups get worse when deep: the flush penalty $P_{misp}$

The picture: a wrong guess means every car currently on the line behind the branch is scrapped. A 5-station line scraps up to 4 cars ( $P_{misp} = 4$ ); a 20-station line scraps up to 19 ( $P_{misp} = 19$ ). Longer line → far bigger loss per wrong guess. This is why CPI climbs with $k$ and why branch prediction accuracy matters so much in deep pipelines.

8. The power symbols: $P_{dynamic}$ , $C$ , $V$ , $f$

$P_{dynamic} = C \cdot V^{2} \cdot f .$

Why this shape? Each switch moves an energy $\propto C V^{2}$ (filling a bucket of size $C$ to pressure $V$ costs energy growing with $V^{2}$ ), and it happens $f$ times per second — energy per event times events per second = power. The $V^{2}$ is why voltage is the strongest lever, tied to Dynamic Voltage and Frequency Scaling (DVFS).

Why the topic needs it: deep pipelines push $f$ up and add $k$ sets of registers (more $C$ ), so power climbs on two fronts. This is the thermal wall Pentium 4 slammed into.

The prerequisite map

Read it top-down: the picture spawns the timing symbols; timing gives $f$ ; depth $k$ and total work $T_{0}$ feed both the timing floor and the hazard cost; $f$ and CPI combine into throughput $R$ ; the whole thing meets the power ceiling — and that collision is the topic.

Equipment checklist

Cover the right side and try to answer each before revealing.

What does

T_{cycle}

measure, and what sets its minimum?

The time between two clock ticks; its minimum is set by the slowest single stage's delay plus register overhead.

Why is

f = 1/ T_{cycle}

rather than something else?

Because "ticks per second" is the reciprocal of "seconds per tick" — pure counting, like laps/second vs seconds/lap.

What is

τ

and what makes it shrink?

The useful logic delay inside one stage; splitting a stage into more, thinner stages shrinks each stage's

τ

What is

T_{0}

and how does it relate to

τ

The total logic delay of the whole instruction in one un-split block; splitting it into

k

equal stages gives

τ = T_{0} / k

What three pieces make up

t_{reg}

, and why won't it shrink with depth?

t_{setup} + t_{clk-to-Q} + t_{skew}

; it is a fixed per-register tax paid every stage no matter how thin the stage.

Write the real cycle time in terms of

τ

and

t_{reg}

T_{cycle} = τ + t_{reg}

k \to \infty

with fixed total work

T_{0}

, what does

T_{cycle} (k)

approach?

t_{reg}

— the register floor; deeper cannot beat it.

What does CPI equal for a perfect pipeline, and why does it rise in deep pipelines?

1

ideally; it rises because deeper pipelines suffer bigger stalls and flushes (bubbles).

How large is the misprediction penalty

P_{misp}

and why does depth hurt?

P_{misp} = k - 1

cycles; a longer pipe holds more wrongly-fetched instructions to scrap on a wrong guess.

Why does raising frequency raise power roughly quadratically?

P_{dynamic} = C V^{2} f

: power scales with

f

directly and with voltage squared, and voltage often must rise to hit higher

f

Combine

f

and CPI into the one true throughput measure

R

R = f / CPI

— frequency alone can lie if CPI worsens.