5.2.9 · D5Processor Datapath & Pipelining
Question bank — Pipeline throughput and CPI
This is a conceptual companion to Pipeline throughput and CPI. No number-crunching here — every item targets a way of thinking that trips people up. Cover the reveal, answer out loud, then check. If an answer surprises you, that misconception was live in your head.
Before we start, three words must mean exactly one thing, because most traps below are really confusions between them:
Keep the performance-metrics identity in view the whole time: Every trap below is really "which of these three terms did the change actually touch?"
True or false — justify
Pipelining reduces the latency of a single instruction.
False. A single instruction still traverses all stages, so its latency is unchanged (often slightly worse due to register overhead ). Only throughput improves.
A 5-stage pipeline always runs 5× faster than the non-pipelined version.
False. Speedup only if stages are perfectly balanced and is negligible. Stage imbalance and register delay typically cap real 5-stage speedup at 3–4×.
If you add more pipeline stages, throughput keeps rising indefinitely.
False. Each new stage adds a register whose overhead becomes a larger fraction of the shrinking clock period. Beyond ~10–15 stages, overhead and extra hazard penalties cause diminishing (then negative) returns.
CPI can never be less than 1.
False. In a scalar pipeline the floor is 1, but a superscalar processor issues several instructions per cycle, so CPI can be well below 1 (e.g. 0.25 = 4 instructions/cycle).
The ideal pipeline CPI of 1 assumes the number of instructions is infinite.
True (in the limit). For instructions the exact CPI is ; the " plus " is the fill cost. It only equals 1 as ; for small the fill penalty is significant.
A faster clock always means a faster processor.
False. Performance = clock rate ÷ CPI. A 5 GHz chip with CPI 4 is slower than a 3 GHz chip with CPI 1. Clock speed is meaningless without CPI.
Forwarding eliminates all data hazards.
False. Forwarding removes hazards whose result is already computed by the time it's needed (EX→EX). A load-use hazard still stalls one cycle because the loaded value isn't ready until after MEM. See 5.2.07-Forwarding-and-bypassing.
Halving one stage's delay by splitting it always improves throughput.
Mostly true, with a catch. It helps only if that stage was the bottleneck (the max). Splitting a non-critical stage changes nothing except adding one more register's overhead — pure loss.
Branch mispredictions raise CPI even if branches are rare.
True but scaled. The penalty term is Branch-rate × Mispredict-rate × Penalty; if branches are rare, the whole product is small, so the CPI hit is small — but never zero unless prediction is perfect.
The clock period of a pipeline equals the sum of all stage delays.
False. That's the single-cycle period. The pipeline period is — only the slowest stage plus register overhead, because stages run concurrently.
Spot the error
"Pipelining a 5-stage design cuts execution time of my one instruction to one-fifth."
The error is confusing latency with throughput. Your one instruction still needs all 5 stages; only the rate of completions for a stream of instructions improves.
"CPI = 1/k for a k-stage pipeline, so a 5-stage pipeline has CPI = 0.2."
Wrong. is the number of stages an instruction passes through (latency, in cycles), not a divisor of CPI. Ideal pipeline CPI is 1 (one completion per cycle), not .
"Adding register overhead to the sum of stage delays gives the pipeline clock period."
Two errors: you use not the sum, and is added to that max once, not to the sum. .
"With CPI = 1.375, we lose 37.5% of throughput versus ideal."
Not exactly — throughput scales as , so actual throughput is of ideal, i.e. a 27% loss, not 37.5%. The CPI increase and the throughput loss are different numbers.
"Structural hazards are the main reason real pipelines miss CPI = 1."
Overstated. In modern designs data hazards and control hazards dominate; structural hazards are largely designed out with duplicated resources and split caches.
"Since forwarding gives EX→EX bypass, load-use dependencies cost zero cycles."
A load's result appears only after the MEM stage, one cycle too late for the dependent's EX. So load-use still costs a one-cycle stall even with full forwarding.
"Speedup from pipelining is exactly the ratio of stages, so 14 stages ⇒ 14× speedup."
Ignores and stage imbalance. Real speedup , which is well below once overhead and imperfect balance are counted.
Why questions
Why do we take the max of stage delays, not the average, for the clock period?
All stages latch on the same clock edge, so the clock must wait for the slowest stage to finish. An average would let fast stages "help" slow ones — but they physically can't; each has its own fixed work.
Why does a bubble (NOP) reduce throughput but not change latency of instructions around it?
A bubble is a wasted cycle where nothing retires, lowering the completion rate. Each real instruction still takes its own stages; the bubble just inserts idle time between completions.
Why is real speedup for a 5-stage pipeline only 3–4× instead of 5×?
Stages are rarely equal, so the clock is set by the slowest (wasting slack in fast stages), and every stage boundary adds overhead. Both erode the theoretical × factor.
Why does improving branch prediction accuracy help throughput even though it doesn't touch the clock?
Better prediction lowers the misprediction stall term, which lowers CPI. Since Time = Instr × CPI × , cutting CPI cuts execution time at the same clock speed.
Why does adding pipeline stages eventually hurt performance?
Each stage adds a register with fixed , so as the clock period shrinks, overhead becomes a bigger fraction of it. Deeper pipelines also make branch penalties (in cycles) larger, raising CPI.
Why is CPI a better performance predictor than raw clock frequency?
Frequency only tells you cycle rate; CPI tells you how many cycles each instruction actually costs. Only their combination (rate ÷ CPI) reflects real work done per second.
Why can a superscalar processor report CPI below 1 while a scalar pipeline cannot?
A scalar pipeline retires at most one instruction per cycle (floor CPI = 1). A superscalar core has multiple execution units and retires several per cycle, so cycles ÷ instructions drops below 1.
Edge cases
What is the CPI of a pipeline executing exactly one instruction?
It's (the fill cost), because that lone instruction must traverse all stages with no overlap to amortize against. The "CPI ≈ 1" claim only holds for long instruction streams.
What happens to throughput if the instruction stream is shorter than the pipeline depth ()?
The pipeline never fully fills before it drains, so overlap is minimal and effective CPI approaches the non-pipelined case — pipelining gives almost no benefit for tiny bursts.
If every instruction stalls one cycle, what is the effective CPI and throughput?
CPI = 1 + 1 = 2, so throughput halves versus ideal — one completion every two cycles. Constant stalls act like doubling the clock period without any of the design benefit.
What is the speedup if all stages have identical delay and ?
Exactly — the ideal upper bound. . Any imbalance or overhead pulls it below.
If branch misprediction rate is 0%, what is the branch contribution to CPI?
Zero. The penalty term is Branch-rate × Mispredict-rate × Penalty; a zero mispredict factor kills the whole term, regardless of how frequent branches are.
What is the throughput of a pipeline whose slowest stage delay grows without bound?
It approaches zero. Since and throughput , an unbounded slow stage means an unbounded clock period and vanishing completion rate — the bottleneck fully dominates.
In the limit and infinitely many perfectly balanced stages, what caps the speedup?
Nothing in this idealized model — speedup . Reality caps it via nonzero , wire delays, and rising hazard penalties, which is why real designs plateau around 10–15 stages.
Recall Self-check: the one-sentence summary
Pipelining trades unchanged latency for higher throughput by overlapping instructions; CPI measures how far real hazards push you above the ideal floor of 1, and Time = Instr × CPI × ties it all together.