Deep pipelining trade-offs: Worked examples — Hardware deep dive

This page is the drill ground for Deep pipelining trade-offs. The parent note built the ideas; here we grind through every kind of number the topic can throw at you — big depths, tiny depths, the degenerate case where extra stages do nothing, the limit as depth runs to infinity, a real chip (Pentium 4), and an exam twist.

Before a single example, one promise: every symbol below was earned in the parent note. Quick refresh, in plain words:

Definition The quantities we juggle

$T_{0}$ — the total logic delay of one instruction with no pipelining, measured in nanoseconds (ns). Picture the whole assembly line collapsed into one giant worker.
$t_{reg}$ — the pipeline-register overhead: the fixed toll (setup time + clock-to-Q + skew) paid every time work is handed across a stage boundary, also in ns. Picture the moment of passing the pizza to the next station — always costs a beat, no matter how small the task.
$k$ — the number of stages we slice the line into (a plain count, no units).
$CPI$ — cycles per instruction: on average, how many clock ticks it takes to complete one instruction. Units: cycles/instruction. In a perfect pipeline one instruction finishes every tick, so $CPI = 1$ ; every stall (a wasted tick spent waiting on a hazard) adds to it. Hence $CPI = 1 + (stall cycles per instruction)$ .

Why frequency is $1/ T_{cycle}$ and not something fancier: frequency literally means "how many cycles fit in one second", and if one cycle lasts $T_{cycle}$ seconds, then $1/ T_{cycle}$ of them fit in a second. If $T_{cycle}$ is in nanoseconds ( $1 0^{- 9}$ s), then $f = 1000/ T_{cycle}$ gives megahertz (MHz), because $1 ns = 1 0^{- 9} s$ and $1 MHz = 1 0^{6}$ ticks/s.

The scenario matrix

Every problem this topic can pose falls into one of these cells. The worked examples below are labelled with the cell(s) they cover, so together they leave no gap.

Cell	What makes it distinct	Covered by
A. Shallow depth	small $k$ (e.g. 5) — registers are a small slice of the cycle	Ex 1
B. Deep depth	large $k$ (e.g. 20) — registers eat a big slice; diminishing returns visible	Ex 2
C. Ratio comparison	"how much better is $k_{2}$ than $k_{1}$ ?" — the marginal question	Ex 3
D. Limiting case	$k \to \infty$ — the hard ceiling on speedup	Ex 4
E. Degenerate input	$t_{reg} = 0$ , or $k = 1$ (no pipeline at all) — the sanity anchors	Ex 5
F. Hazards fold in	branches raise CPI; frequency win is partly eaten	Ex 6
G. Imbalanced stages	the "just divide evenly" trap — longest stage rules	Ex 7
H. Real-world word problem	Pentium 4: frequency up but net performance flat	Ex 8
I. Power / DVFS twist	dynamic power scaling — exam-style curveball	Ex 9

We reuse one baseline machine unless a cell says otherwise: $T_{0} = 10 ns, t_{reg} = 0.2 ns .$

Example 1 — Shallow depth (Cell A)

Forecast: Guess first — will the speedup be close to $5 \times$ , or noticeably less? Jot a number.

Stage logic delay $τ = T_{0} / k = 10/5 = 2 ns$ . Why this step? Even splitting means each stage gets an equal $1/ k$ share of the work; $τ$ is that share and it is what sets the useful part of the cycle.
Cycle time $T_{cycle} = τ + t_{reg} = 2 + 0.2 = 2.2 ns$ . Why this step? A cycle can't be shorter than the slowest stage plus the toll to cross into the next stage — you always pay the register overhead.
Frequency $f = 1000/ T_{cycle} = 1000/2.2 \approx 454.5 MHz$ . Why this step? We convert "one tick takes 2.2 ns" into "how many ticks per second"; the $1000$ turns ns into MHz.
Speedup $= T_{0} / T_{cycle} = 10/2.2 \approx 4.55 \times$ . Why this step? The unpipelined machine finishes one instruction every $10$ ns; the pipeline finishes one every $2.2$ ns once the pipe is full.

Verify: Register overhead as a fraction of the cycle is $0.2/2.2 \approx 9%$ — small, as promised for shallow pipelines. And $4.55 < 5$ , so we did not get the ideal $5 \times$ : the $0.2$ ns toll cost us the missing $0.45 \times$ . Sanity: $4.55 \times 2.2 = 10.0$ ns ✓.

Example 2 — Deep depth (Cell B)

Forecast: We quadrupled the depth from Ex 1 (5→20). Will speedup quadruple to $\approx 18 \times$ ? Guess.

$τ = 10/20 = 0.5 ns$ . Why? Each stage now does a quarter of what it did at $k = 5$ .
$T_{cycle} = 0.5 + 0.2 = 0.7 ns$ . Why? Same toll ( $0.2$ ), but now it's a big slice of a tiny cycle.
$f = 1000/0.7 \approx 1428.6 MHz$ . Why? Convert to ticks/s.
$Speedup = 10/0.7 \approx 14.29 \times$ . Why? Throughput ratio vs. unpipelined.

Verify: Overhead fraction is now $0.2/0.7 \approx 29%$ — over triple the $9%$ of Ex 1. That's exactly why going 5→20 stages (a $4 \times$ deepening) only bought $14.29/4.55 \approx 3.14 \times$ more speedup, not $4 \times$ . The register toll is stealing more of every cycle. ✓

Example 3 — Marginal comparison (Cell C)

Forecast: Doubling stages — does frequency double?

$τ = 10/10 = 1 ns$ , so $T_{cycle} = 1 + 0.2 = 1.2 ns$ . Why? Same recipe.
$f_{10} = 1000/1.2 \approx 833.3 MHz$ . Why? Ticks/s.
$S_{10} = 10/1.2 \approx 8.33 \times$ . Why? Speedup vs. unpipelined.
Ratio to Ex 1: $S_{10} / S_{5} = 8.33/4.55 \approx 1.83 \times$ . Why this step? This is the honest payoff of doubling depth — the number a designer actually cares about.

Verify: We doubled $k$ but got only $1.83 \times$ , not $2 \times$ . The missing $0.17 \times$ is the register toll asserting itself — and Figure 1 below shows exactly where this point sits on the curve. ✓

Reading Figure 1 (tie it to the numbers above): the dashed yellow line is the ideal speedup $= k$ (what you'd get if $t_{reg} = 0$ , our Ex 5a result). The solid blue curve is the real speedup from our formula. Find the three white dots: at $k = 5$ it reads $4.5 \times$ (Ex 1), at $k = 10$ it reads $8.3 \times$ (this example), at $k = 20$ it reads $14.3 \times$ (Ex 2). Watch the vertical gap between blue and yellow widen as you move right — that widening gap is the register toll, and it is why the marginal ratio fell from an ideal $2 \times$ to our $1.83 \times$ . The pink dotted line at $50$ is the ceiling from Ex 4 that the blue curve creeps toward but never touches.

Example 4 — The limit (Cell D)

Forecast: Does speedup grow forever, or hit a wall? What number?

Start from $Speedup (k) = \frac{T _{0}}{\frac{T _{0}}{k} + t _{reg}}$ . Why? This is our exact formula — no approximation yet.
As $k \to \infty$ , the term $\frac{T _{0}}{k} \to 0$ . Why this step? Dividing a fixed $T_{0} = 10$ by a huge $k$ shrinks it toward zero — each stage's useful work vanishes.
So $Speedup \to \frac{T _{0}}{t _{reg}} = \frac{10}{0.2} = 50 \times$ . Why? With the logic term gone, only the toll remains in the denominator, and it's a fixed $0.2$ .

Verify: No matter how many stages you build, you can never beat $50 \times$ on this machine. Check with a huge $k$ : at $k = 1000$ , $T_{cycle} = 0.01 + 0.2 = 0.21$ , speedup $= 10/0.21 \approx 47.6 \times$ — creeping toward 50 but never reaching it. On Figure 1 this is the pink dotted asymptote. This is the register-limited regime the parent note warned about. ✓

Example 5 — Degenerate inputs (Cell E)

Forecast: In (a), what should speedup equal exactly? In (b), should there be any speedup?

Case (a): $t_{reg} = 0$

$T_{cycle} = T_{0} / k + 0 = 10/20 = 0.5 ns$ . Why? Remove the toll entirely.
$Speedup = 10/0.5 = 20 \times$ . Why this step? With zero overhead, speedup equals $k$ exactly — the ideal the dashed yellow line in Figure 1 represents.

Case (b): $k = 1$

$T_{cycle} = 10/1 + 0.2 = 10.2 ns$ . Why? One "stage" holds all the logic, yet we still pay one register toll to latch the result.
$Speedup = 10/10.2 \approx 0.98 \times$ . Why this step? Adding a register but no splitting makes you slightly slower than unpipelined — a pure loss.

Verify: (a) confirms $t_{reg} = 0 \Rightarrow speedup = k$ , the theoretical best. (b) confirms $k = 1$ is never worth it (0.98 < 1) — pipelining only pays once $k \geq 2$ and the frequency gain outweighs the one-time toll. Both anchors behave. ✓

Example 6 — Hazards fold in (Cell F)

Forecast: Frequency is $\sim 3 \times$ higher at 20 stages. Will effective speedup also be $\sim 3 \times$ ? Or less?

Mispredictions per instruction $= 0.1 \times \frac{1}{5} = 0.02$ . Why? $\frac{1}{5}$ of instructions are branches; $10%$ of those are mispredicted.
CPI (5-stage) $= 1 + 0.02 \times (5 - 1) = 1 + 0.08 = 1.08$ . Why this step? Each misprediction flushes $k - 1 = 4$ stages; multiply by how often it happens and add to the base CPI of 1.
CPI (20-stage) $= 1 + 0.02 \times (20 - 1) = 1 + 0.38 = 1.38$ . Why? Now $k - 1 = 19$ instructions get flushed per misprediction — nearly $5 \times$ the pain.
Effective speedup ratio $\frac{f _{20}}{f _{5}} \times \frac{CPI _{5}}{CPI _{20}} = \frac{1428.6}{454.5} \times \frac{1.08}{1.38}$ . Why this step? Performance $= f / CPI$ ; the ratio of performances is (frequency ratio) × (inverse CPI ratio).
Compute: $3.143 \times 0.7826 \approx 2.46 \times$ . Why? The raw frequency gain of $3.14 \times$ is dragged down to $2.46 \times$ by the worse CPI.

Verify: Frequency alone said $3.14 \times$ ; hazards cut it to $2.46 \times$ . The CPI penalty ( $1.38/1.08 = 1.28$ ) ate about $22%$ of the frequency win. This is why Branch Prediction accuracy is life or death in deep pipelines — see also Pipeline Hazards. ✓

Example 7 — Imbalanced stages (Cell G)

Forecast: The average stage is $1.0$ ns. Does the clock run at the average?

Find the longest stage. The maximum of the list is $1.5 ns$ . Why this step? Every stage must finish before the clock ticks; the clock waits for the slowest worker, never the average.
$T_{cycle} = 1.5 + 0.2 = 1.7 ns$ . Why? Longest stage plus the fixed register toll.
$Speedup = 10/1.7 \approx 5.88 \times$ . Why? Throughput vs. unpipelined.
Compare to balanced Ex 3 ( $8.33 \times$ ). Why this step? Shows how much imbalance costs.

Verify: The naive even-split predicted $8.33 \times$ ; the real imbalanced pipeline delivers only $5.88 \times$ — a loss of about $29%$ , caused entirely by the one $1.5$ ns stage. The sum of delays still equals $\sum = 10.0$ ns (average $1.0$ ), yet the max is what rules. Never average — always take the max. ✓

Reading Figure 2 (tie it to the numbers above): each blue bar is one stage's logic delay; the $x$ -axis is the stage number, the $y$ -axis is delay in ns. The single pink bar is stage 4 at $1.5$ ns — the tallest. The pink dashed line drawn across at $1.5$ is what actually sets $T_{cycle}$ (add $t_{reg}$ to get $1.7$ ). The yellow dotted line at $1.0$ marks the average — notice it sits below the pink line, which is the whole point: the clock obeys the tallest bar, so every shorter stage (all the ones under the pink line) sits idle for the leftover time, wasting it. That wasted gap is why speedup fell from $8.33 \times$ to $5.88 \times$ .

Example 8 — Real-world word problem: Pentium 4 (Cell H)

Forecast: $50%$ more frequency — surely faster?

Performance = $f / CPI$ . For Northwood: $Perf_{N} = f /1.0 = f$ . Why? Throughput is ticks/s divided by cycles/instruction.
Prescott: $Perf_{P} = 1.5 f /1.5 = f$ . Why this step? Both factors moved by exactly $1.5 \times$ — one helping, one hurting.
Ratio $Perf_{P} / Perf_{N} = f / f = 1.0 \times$ . Why? The gains cancelled precisely.

Verify: Frequency rose $1.5 \times$ but performance rose $1.0 \times$ — zero net gain, exactly the parent note's "deep-pipeline wall." Meanwhile dynamic power (Ex 9) rose, so Prescott was worse per watt. Intel's answer: abandon deep pipes for the 14-stage Core design with wider issue (Superscalar Architectures) and better IPC. ✓

Example 9 — Power / DVFS twist (Cell I)

Forecast: Frequency rose $\sim 1.7 \times$ — is that the whole power story?

Frequency ratio $\frac{f _{20}}{f _{10}} = \frac{1428.6}{833.3} \approx 1.714$ . Why? From Examples 2 and 3.
Power ratio $\frac{P _{20}}{P _{10}} = \frac{C _{20}}{C _{10}} \cdot \frac{V ^{2}}{V ^{2}} \cdot \frac{f _{20}}{f _{10}} = 1.1 \times 1 \times 1.714$ . Why this step? $V$ cancels (held constant); capacitance and frequency each multiply in.
Evaluate: $1.1 \times 1.714 \approx 1.886$ . So dynamic power rises by a factor of $\approx 1.89 \times$ . Why this step? We actually multiply out the numbers to get the final answer — no leaving it symbolic.

Verify: Power grew by $\approx 1.89 \times$ while performance (roughly the frequency, $1.71 \times$ ) grew less — so power grew faster than performance ( $1.89 > 1.71$ ), meaning worse efficiency (more watts per instruction). This is precisely why real chips lower $V$ as they add stages, using DVFS; because $P \propto V^{2}$ , a small voltage drop recovers a lot of power. And clock distribution to $2 \times$ the registers is itself a major power sink (30–40% in Pentium 4). ✓

Recall Self-test — cover the answers

Even 5→20 stages gives what speedup ratio (register-limited)? ::: About $3.14 \times$ , not $4 \times$ (Ex 2 ÷ Ex 1). The absolute speedup ceiling on the baseline machine is? ::: $T_{0} / t_{reg} = 10/0.2 = 50 \times$ (Ex 4). With $k = 1$ and a real register, speedup is? ::: $\approx 0.98 \times$ — slightly worse than unpipelined (Ex 5b). What sets the cycle time in an imbalanced pipeline? ::: The longest stage plus $t_{reg}$ , never the average (Ex 7). Why did Prescott's $1.5 \times$ frequency give $1.0 \times$ performance? ::: CPI also rose $1.5 \times$ , cancelling the gain (Ex 8). By what factor did dynamic power rise in Ex 9? ::: $\approx 1.89 \times$ , more than the $1.71 \times$ frequency gain — worse efficiency.

Connections: Instruction Pipelining Basics · Pipeline Hazards · Branch Prediction · Superscalar Architectures · Clock Distribution and Skew · Dynamic Voltage and Frequency Scaling (DVFS) · Pentium 4 vs Core Microarchitecture · 5.2.11 Deep pipelining trade-offs (Hinglish)