Deep pipelining trade-offs: Question bank — Hardware deep dive

This is a concept-trap question bank for the parent topic. Each item below is a one-line reveal: read the prompt, answer it in your head, then peek. The goal is not arithmetic (that lives in the worked-example pages) — it is catching the wrong intuitions that deep pipelining quietly invites.

Before we start, the handful of symbols we lean on constantly, in plain language:

Stage — one work-station on the assembly line; one instruction sits in one stage per clock tick.
$T_{0}$ (un-pipelined total logic delay) — the time it takes to do all the instruction's combinational work in one long block, before we cut it into stages. Think of it as the whole pizza-making time end to end, with no hand-offs.
$k$ — the number of stages we cut $T_{0}$ into.
$τ$ (per-stage logic delay) — the combinational work left in one stage. If we split $T_{0}$ evenly, then $τ = T_{0} / k$ ; we also write $τ_{logic}$ for the same thing when emphasising it is logic only, not register cost.
$t_{reg}$ (register overhead) — the fixed per-stage tax to hand work between stages: $t_{reg} = t_{setup} + t_{clk-to-Q} + t_{skew} + t_{jitter}$ .
Cycle time ( $T_{cycle}$ ) — how long one tick lasts, set by the slowest stage's logic plus that overhead: $T_{cycle} = τ + t_{reg}$ .
$f$ (clock frequency) — how many ticks happen per second; it is simply one divided by the cycle time, $f = 1/ T_{cycle}$ . A shorter cycle means a bigger $f$ .
CPI — cycles per instruction; how many ticks, on average, each instruction really costs once stalls and flushes are counted.
IPC (instructions per cycle) — the flip side of CPI, $IPC = 1/ CPI$ ; how much useful work finishes per tick. Deep pipelines tend to raise $f$ but lower IPC.
Throughput — real delivered performance, equal to $f / CPI$ (equivalently $f \cdot IPC$ ). Both $f$ and CPI matter, which is the whole moral of this page.
$C$ (load capacitance) and $V$ (supply voltage) — in the dynamic-power law $P \propto C V^{2} f$ , $C$ is how much electrical charge the circuit must swing each switch (more registers means more $C$ ) and $V$ is the voltage it swings through.

Figure 1 (below) plots that speedup curve. Trace it with your finger: the magenta line climbs steeply at small $k$ , then bends and hugs the violet dashed ceiling $T_{0} / t_{reg}$ , while the orange dotted "ideal" line (the fantasy where $t_{reg} = 0$ ) keeps rising forever. The gap between magenta and orange is the register tax. Every "diminishing returns" answer on this page is really pointing at that bend.

Figure 2 (below) shows the other half of the story. Each bar is how many instructions a single wrong branch guess throws away — height $k - 1$ . Watch it rise as a straight line: 4 flushed at 5 stages, 30 at 31. Frequency (Figure 1) gives, hazards (Figure 2) take back — and the taking grows faster and faster with depth.

True or false — justify

A 20-stage pipeline runs 4× faster than a 5-stage one at the same logic

False — frequency rises only if the slowest stage shrinks 4×, and register overhead

t_{reg}

is a fixed tax that eats a bigger share as stages thin, so the speedup is well under 4× (this is exactly the bend in Figure 1).

Making each stage do less work always shortens the clock period

False — it shortens the logic part

τ

, but

t_{setup} + t_{clk-to-Q} + t_{skew}

stay put, so once

τ

is tiny the period stops shrinking (register-limited regime — the flat part of Figure 1).

The purely analytical model says "deeper is always better"

True — differentiating the cycle-time function

T_{cycle} (k) = T_{0} / k + t_{reg}

with respect to

k

gives

\frac{d T _{cycle}}{d k} = - T_{0} / k^{2}

, which is always negative, so period keeps falling; the model is wrong because it ignores hazards and power, not because the math is.

Branch misprediction penalty is a fixed 4 cycles regardless of depth

False — the penalty is roughly

k - 1

cycles because you flush every instruction fetched behind the branch; a 20-stage pipeline throws away ~19, a 31-stage one ~30 (this is Figure 2's straight climb).

Doubling frequency doubles throughput, all else equal

True as a principle — throughput is

f / CPI

, so if CPI is held fixed, doubling

f

does double it; the trap is that deep pipelining rarely holds CPI fixed (more stalls and flushes push CPI up), so in practice the doubling is partly cancelled.

Adding pipeline registers costs power even when the pipeline is idle

True — the clock network must still toggle all

k

Perfect stage balance is achievable if you just divide the logic evenly

False — some operations are atomic (a cache access, a TLB lookup) and cannot be cut mid-way, so real stage delays are uneven and the longest one sets the clock.

Higher clock frequency alone made Pentium 4 Prescott faster than Northwood

False — frequency rose ~1.5× but CPI worsened ~1.5× from the longer misprediction penalty, so per-clock performance

f / CPI

landed near 1.0× — essentially flat.

With perfect branch prediction, deep pipelining has no downside

False — even at 100% prediction you still pay register overhead, stage-imbalance waste, and quadratically-scaling clock/register power.

The register overhead

t_{reg}

includes clock skew

True —

t_{reg} = t_{setup} + t_{clk-to-Q} + t_{skew} + t_{jitter}

; skew is the arrival-time mismatch of the clock edge across registers, a real fixed cost each stage must budget for.

Spot the error

"10 ns logic split into 10 stages gives exactly 1 ns cycle time."

Error — with

T_{0} = 10

ns and

k = 10

the logic per stage is

τ = T_{0} / k = 1

ns, but the cycle is

τ + t_{reg} = 1 + t_{reg}

; also real logic won't split into ten equal 1 ns pieces, so the longest stage governs.

"Deeper pipelines are always slower because hazards dominate."

Error — deeper pipelines usually raise frequency; whether they are net slower depends on whether the CPI penalty outweighs that frequency gain, which is a trade-off, not a guarantee.

"Speedup from a

k

-stage pipeline approaches

k

k \to \infty

Error — with even splitting

τ = T_{0} / k

Speedup = \frac{T _{0}}{T _{0} / k + t _{reg}}

approaches

T_{0} / t_{reg}

, a fixed ceiling set by register delay, not

k

(magenta bends away from orange in Figure 1).

"A data hazard in a 31-stage pipeline is the same 1–2 cycle stall as in a 5-stage one."

Error — an operand may not be ready for many more stages downstream, so bypassing gets more complex and residual stalls grow; the 1–2 cycle figure is a 5-stage artifact.

"Reducing voltage cancels the extra power of a deeper pipeline entirely."

Error — in the dynamic-power law

P \propto C V^{2} f

(with

C

= load capacitance,

V

= supply voltage), lowering

V

helps, but the added registers raise

C

and the higher $f$ raises

f

, so it only partly offsets a genuinely larger power draw — the technique of trading

V

against

f

is DVFS, Dynamic Voltage and Frequency Scaling, covered in Dynamic Voltage and Frequency Scaling (DVFS).

"CPI of 1 means the pipeline is optimal."

Error — CPI counts cycles per instruction but ignores how long each cycle is; a slow, low-CPI design can lose to a fast, higher-CPI one because performance is

f / CPI

"Forwarding paths grow linearly with pipeline depth."

Error — the number of potential source-to-destination bypasses scales like

O (k^{2})

, which is a big reason deep pipelines are hard to verify.

Why questions

Why does register overhead cap the speedup instead of hazards, in the idealized model?

Because the ideal model assumes no hazards and perfect balance, the only surviving fixed cost is

t_{reg}

— so as

k

grows the logic term

T_{0} / k

vanishes and the curve flattens onto the violet ceiling

T_{0} / t_{reg}

in Figure 1; hazards only enter the realistic model.

Why did Intel abandon the deep-pipeline path for the Core architecture?

Because Prescott hit the "deep-pipeline wall" you can read off both figures at once — Figure 1's frequency gains had flattened while Figure 2's flush penalty ballooned to ~30, so falling IPC plus ~115 W power gave near-zero net gain; Core went shallower (~14 stages) but wider to raise IPC — see Pentium 4 vs Core Microarchitecture.

Why does a deeper pipeline make Branch Prediction more important, not less?

Because the misprediction penalty is the height of the bar in Figure 2 (

k - 1

), each wrong guess costs far more as depth grows, so the same accuracy silently loses more throughput — better prediction is needed just to stand still.

Why do memory stages resist being sliced thinner?

Because a cache or TLB access is atomic and variable-latency (a hit and a miss take very different times); you cannot pause halfway through an access, so that stage forms a hard floor on how short a stage can be, capping how far right you can usefully travel along Figure 1.

Why is "more stages = higher IPC" the wrong expectation?

Because extra stages raise frequency (

f

, Figure 1), not IPC; IPC usually falls with depth as the growing flush penalty (Figure 2) and extra stalls push CPI up — exactly what dropped Prescott's IPC from ~1.5 to ~1.0.

Why does the clock network dominate power in very deep pipelines?

Because every one of the

k

f

, so in

P \propto C V^{2} f

both the capacitance

C

(more registers) and

f

climb together — see Clock Distribution and Skew.

Why can a superscalar design beat a deeper scalar one at the same frequency?

Because issuing several instructions per cycle attacks CPI directly (raising IPC, more useful work per tick), whereas deepening the pipe attacks

f

while worsening CPI — often a better trade in the

f / CPI

balance.

Edge cases

What happens to speedup when

t_{reg} = 0

Then

T_{cycle} = τ = T_{0} / k

and

Speedup = k

exactly — this is the orange dotted "ideal" line in Figure 1 that never bends — which is why

t_{reg}

is the single term that pulls the real (magenta) curve down toward its ceiling.

What is the pipeline period when logic delay

τ_{logic} \to 0

It flattens to

T_{cycle} \approx t_{reg}

— the register-limited regime, the flat right end of Figure 1 — so adding more stages gains essentially nothing.

For a 2-stage pipeline (

k = 2

) versus un-pipelined, does the formula still show a gain?

Yes —

Speedup (2) = \frac{2 T _{0}}{T _{0} + 2 t _{reg}}

, which is above 1 whenever

t_{reg} < T_{0} /2

; but note a "1-stage pipeline" (

k = 1

) is just the un-pipelined logic plus one register, so it is slightly slower — pipelining only pays once

k \geq 2

If branch prediction were perfect (100%), does the control-hazard penalty vanish for any

k

Yes, the misprediction term (the Figure 2 bars) goes to zero regardless of depth — but data hazards, stage imbalance, and register/power costs remain, so depth still has limits.

Where does the empirical sweet spot

k^{*} \approx 10

–15 come from?

It is where the two figures cross in importance: by

k \approx 10

–15 Figure 1's frequency curve has already captured most of its 2–3× gain (register overhead is a tolerable ~10–20% of the cycle), while Figure 2's flush penalty is still small enough that a 95%+ predictor keeps CPI in check — push past that and hazard/power costs grow faster than the flattening frequency benefit, so measured throughput

f / CPI

turns over. It is an empirical balance of real designs, not a clean closed-form root.

What throughput trend appears as

k

grows past that sweet spot?

Frequency gains flatten (Figure 1's ceiling) while hazard penalties keep climbing (Figure 2's straight line), so throughput

f / CPI

peaks and then decreases — the diminishing-then-negative returns region.

Recall Fastest gut-check

Real performance is ==throughput $= f / CPI$ == (with $f = 1/ T_{cycle}$ the clock frequency), not $f$ alone — deep pipelining always raises the numerator but usually raises the denominator too, and register overhead caps how far the numerator can go via $Speedup \to T_{0} / t_{reg}$ .