Classic 5-stage pipeline (IF - ID - EX - MEM - WB): Visual walkthrough

We will keep three plain-word symbols throughout. Let us name them before any picture, so nothing is ever a surprise:

Everything below is counting ticks and multiplying by $T_{c l k}$ . That is the whole derivation.

Step 1 — One instruction is a chain of $k$ jobs

WHAT. Take a single instruction. It cannot be done in one indivisible blob — it is really a sequence of smaller jobs that must happen in order: fetch it, then decode it, then compute, then touch memory, then write the answer back.

WHY. Each job needs the previous job's output. You cannot decode an instruction you have not fetched; you cannot add numbers you have not read. This forced order is what makes a line (an assembly line) the natural shape — see the Register File read in ID feeding the ALU Design in EX.

PICTURE. Below, one instruction flows left-to-right through $k = 5$ boxes. Each box is one job; the arrows are "hand the result forward."

Figure — Classic 5-stage pipeline (IF - ID - EX - MEM - WB)

Step 2 — The non-pipelined machine: no overlap allowed

WHAT. Now run $N$ instructions on a machine that refuses to overlap them: instruction 2 may not start box 1 until instruction 1 has fully left box $k$ .

WHY. This is our baseline — the thing pipelining must beat. If we do not know what "slow" looks like, "fast" means nothing.

PICTURE. Each instruction occupies the whole line by itself, then the next begins. Notice the huge white gaps: while instruction 1 is in "compute," the "fetch" box sits idle.

Step 3 — Overlap them: a new instruction every tick

WHAT. Keep the same boxes, but the moment instruction 1 leaves box 1 (fetch), let instruction 2 enter box 1. Every tick, everybody shifts one box to the right, and a fresh instruction slides in at the left.

WHY. Those idle boxes in Step 2 were wasted hardware. Overlapping fills them: at full speed all $k$ boxes are busy on $k$ different instructions at once. This is the assembly line. The little trays that hold each instruction's half-done state between boxes are the pipeline registers (IF/ID, ID/EX, …).

PICTURE. This is the classic staircase diagram. Read a column (one tick) to see all boxes busy; read a row (one instruction) to see its 5-box journey.

Step 4 — Counting ticks: the "fill" then "one-per-tick"

WHAT. Count how many ticks the whole staircase takes. Two phases:

Fill: the very first instruction still needs its full $k$ ticks to reach the far right and finish. First result appears at tick $k$ .
Steady state: after that first result, one more instruction pops out the right end every single tick. There are $N - 1$ instructions left to finish.

WHY. We are literally reading the staircase figure. The first finish is delayed (the pipe must fill); every finish after that is spaced one tick apart because the whole line advances together.

PICTURE. The staircase again, now annotated: the orange bracket marks the $k$ fill ticks, the magenta bracket marks the $N - 1$ one-per-tick finishes.

Step 5 — Divide the two times: the speedup formula appears

WHAT. Put the baseline (Step 2) over the overlapped time (Step 4). Speedup = "how many times faster."

WHY. "Faster by a factor of $X$ " means old-time divided by new-time. Nothing more.

PICTURE. Two bars, same instructions: the tall $T_{serial}$ bar over the short $T_{pipe}$ bar; the ratio between their lengths is the speedup.

Step 6 — The limit: why it caps at $k$ (large $N$ )

WHAT. Push $N$ to be enormous (millions of instructions). Watch the fraction settle.

WHY. Real programs run billions of instructions, so the large- $N$ behaviour is the one that matters. We divide top and bottom by $N$ to see what survives.

$\frac{N k}{k + N - 1} = \frac{k}{\frac{k}{N} + 1 - \frac{1}{N}} N \to \infty \frac{k}{0 + 1 - 0} = k$

$\frac{k}{N}$ and $\frac{1}{N}$ — the "fill cost" spread over $N$ instructions; as $N$ grows they shrink to $0$ .
What remains — just $k$ . The pipeline can be at most $k$ times faster, because at best $k$ instructions overlap.

PICTURE. A curve of speedup versus $N$ : it climbs and flattens toward the dashed line $y = k$ , never crossing it.

Step 7 — Edge & degenerate cases (the corners you must not trip on)

WHAT / WHY / PICTURE for each corner, so no scenario surprises you:

The annotated figure shows all three corners side by side.

The one-picture summary

Everything above collapses into a single diagram: the staircase (overlap), the two brackets ( $k$ fill, $N - 1$ steady), the two comparison bars, and the ratio that is the speedup — with the $N \to \infty$ limit $k$ pinned in the corner.

Recall Feynman retelling — the whole walkthrough in plain words

One instruction is really five little jobs done in a row, so alone it takes 5 ticks (Step 1). If you make each instruction finish completely before starting the next, running 1000 of them costs $1000 \times 5$ ticks — and most boxes sit idle the whole time (Step 2). So instead you slide a new instruction into the first box the instant it frees up: now all five boxes work at once on five different instructions (Step 3). Counting ticks: the first result waits 5 ticks for the pipe to fill, then one result pops out every tick for the remaining 999 (Step 4). Divide the slow time by the fast time and the tick-length cancels, leaving $\frac{N k}{k + N - 1}$ (Step 5). For huge programs that fraction climbs toward $k$ but never passes it, because at best only $k$ instructions overlap (Step 6). And in the corners — one instruction, one stage, or uneven boxes — the gain vanishes or shrinks, which is exactly what the formula predicts (Step 7).

Active recall

Where does the numerator

N k

in the speedup come from?

The no-overlap baseline time:

N

instructions each costing

k

ticks (Step 2).

Where does the denominator

k + N - 1

come from?

The overlapped run:

k

ticks to fill the pipe, then

N - 1

more instructions finishing one per tick (Step 4).

Why does

T_{c l k}

vanish from the speedup?

It appears in both the serial and pipelined times, so it cancels — the speedup shape depends only on

N

and

k

What is the speedup when

N = 1

, and why?

Exactly 1 — a single instruction has nothing to overlap with.

Why does the speedup cap at

k

N \to \infty

The fill terms

\frac{k}{N}

and

\frac{1}{N}

shrink to 0, leaving

k

; at best only

k

instructions overlap.