ARM architecture overview: Visual walkthrough — Hardware deep dive

Everything below assumes only one thing: a computer runs instructions one after another, top to bottom, like reading a list. We build the rest.

Step 1 — What a pipeline actually is

WHAT. A modern CPU does not finish one instruction before starting the next. It works like a laundromat: while shirt #1 is drying, shirt #2 is already washing. Each instruction passes through stages. In the classic ARM7 there are three:

Fetch (F) — grab the next instruction from memory.
Decode (D) — figure out what it means (which operation, which registers).
Execute (E) — actually do the arithmetic.

WHY. If each stage takes one cycle (one tick of the CPU clock), doing them one-at-a-time wastes time: the "fetch" hardware sits idle while "execute" runs. Overlapping them keeps every piece of hardware busy every cycle.

PICTURE. Look at the coloured diagonal below. Read each row as one clock cycle (time going down). At the bottom you see the payoff: after the pipe is full, one instruction finishes every single cycle, even though each one personally takes 3 cycles.

Step 2 — A branch is a fork in the road

WHAT. A branch is an instruction that says "maybe jump somewhere else." Example:

CMP r0, #0      ; compare r0 to 0, record the result in flags
JLE skip        ; Jump if Less-or-Equal — go to 'skip' if r0 <= 0
ADD r1, r1, #1  ; otherwise run this
skip: ...

WHY it is dangerous. Remember from Step 1: the CPU fetches the next instruction before the current one finishes. But which instruction is next after JLE? That depends on CMP's result — which hasn't finished executing yet! The fetch stage has to guess.

PICTURE. The road splits. The CPU is standing at the fork with its eyes closed, and it must start walking down one path before the signpost (CMP's flags) is readable.

Step 3 — The guess, and the cost of guessing wrong

WHAT. The CPU uses a branch predictor: a piece of hardware that bets on "taken" or "not taken" based on past history. If it bets right, almost no time is lost. If it bets wrong, every instruction it fetched down the wrong path is garbage and must be thrown away — a pipeline flush.

WHY it hurts so much. In Step 1 the pipe was full of useful work. A flush empties it. The CPU must re-fetch from the correct path and refill every stage from scratch — cycles of pure waste.

PICTURE. The predictor guessed "go right", filled the pipe with red (wrong-path) instructions, then discovered the truth was "go left". Watch all the red work get dumped into the trash, and the pipe start over.

Definition Flush penalty (and why it is only

roughly the pipeline depth) A first-order estimate is: $C_{flush} \approx (pipeline stages between Fetch and where the branch is resolved)$

$C_{flush}$ :: cycles wasted per mispredict.
Short 3-stage ARM7 → small penalty (~2). Deeper cores → bigger.

But this is a floor, not the truth. The real penalty is shaped by hardware the simple formula ignores:

A branch-target buffer (BTB) can supply the target during Fetch, hiding much of the cost of a correctly predicted taken branch.
Decode/dispatch timing and partial stage overlap mean the branch may resolve earlier or later than "one stage per cycle" suggests.
On out-of-order cores, recovery also involves rewinding the register-rename map and reclaiming issue slots, which can add cycles beyond the raw depth.

Because of all this, published mispredict penalties on real Cortex-A designs commonly sit in the ~15–20 cycle range, not a single universal number.

Step 3½ — Even a correct guess is not perfectly free

WHAT. The clean claim "correct prediction costs 0" is a simplification. On real pipelines, a correctly predicted taken branch often still costs a small amount: at least the resolution latency (the branch must be recognised and its target formed), and if the BTB misses on the first sighting, a 1–2 cycle bubble while the front-end redirects.

WHY it matters. It means branches are never truly free, so our model has a hidden third term. Let $C_{ok}$ be the small penalty of a correctly predicted branch (often $\approx 1$ cycle for a taken branch, $\approx 0$ for a not-taken one).

PICTURE. The green "correct" slice from Step 4 is not flat on the floor — it sits a hair above zero. Small, but honest.

Step 4 — The average cost of a branch (dominant term)

WHAT. Not every branch mispredicts. Suppose the predictor is right 85% of the time. Then a fraction $P_{mispredict} = 0.15$ of branches pay the full flush price; the rest pay only the tiny $C_{ok}$ from Step 3½. Ignoring that small toll for a moment, the average extra cost is dominated by the mispredict term.

WHY multiply, not add. We want the expected waste across many branches. Probability $\times$ penalty is exactly what "expected cost" means: a rare disaster and a common near-free pass, blended into one number.

PICTURE. A bar split into two slices — the tall-but-thin red slice (rare, costs 40) and the wide green slice (common, costs the small $C_{ok} \approx 1$ , drawn just above the floor). The dashed line is the average height: the effective cost every branch "owes".

So on average, every branch costs roughly 6 wasted cycles in this model. Now ARM asks: can we avoid the branch entirely?

Step 5 — ARM's trick: tag the instruction, don't jump

WHAT. Every ARMv7 instruction carries a 4-bit condition field in bits 31–28 (see the encoding table in the parent note). The instruction runs only if the flags match; otherwise it silently becomes a NOP (no-operation — it just occupies its one cycle and does nothing).

CMP r0, #0        ; set flags from r0 - 0
ADDGT r1, r1, #1  ; ADD, but only 'if Greater Than'  — else a 1-cycle NOP
MOVLE r2, #0      ; MOV,  only 'if Less-or-Equal'     — else a 1-cycle NOP

WHY this dodges the whole problem. There is no fork in the road. The instructions after CMP are always the next ones in memory — the fetch stage never has to guess. The condition is checked late, inside the pipeline, and just decides whether the result is written or dropped. No wrong path is ever fetched → no flush is possible.

PICTURE. Compare the two timelines side by side. Top (branch version): a fork, a wrong guess, a red flush. Bottom (conditional version): one straight pipe, some instructions greyed-out as NOPs, but never a bubble.

Step 6 — The break-even line (every case covered)

WHAT. When does predication actually win? Let a conditional block contain $n$ instructions, of which a false condition would waste $n$ NOP cycles. Compare that to branching around them at cost $C_{branch} \approx 6$ .

WHY draw a line. We must cover all cases, not just the happy one. Small $n$ → predication wins. Large $n$ → branch wins. There is a crossover.

PICTURE. Two cost lines vs block size $n$ : the flat branch cost (about 6, once a mispredict happens) and the rising predication cost ( $n$ NOP cycles). Where they cross is the decision boundary.

The one-picture summary

The whole chain in one image: a full pipe (IPC = 1) → a branch forks it → a wrong guess flushes it (and even a right guess costs a small toll) → averaged, that costs about 6 cycles in our teaching model → ARM replaces the fork with a tagged straight-line pipe that can NOP but never bubble → but for big blocks the NOP tax exceeds the branch, which is why deep, well-predicted pipelines eventually retired the trick.

Recall Feynman retelling — say it in plain words

A CPU reads instructions like an assembly line: while one is being done, the next is already being fetched. That only works if it knows what comes next. A branch breaks that — it's a fork, and the machine has to guess which way to go before the answer is ready. Guess right, and it costs almost nothing (a small resolution/target toll, maybe 1 cycle). Guess wrong, and it must dump everything it started and refill the line — on a deep out-of-order core that's roughly 15–40 cycles depending on the design. Since guesses are wrong maybe 15% of the time, each branch quietly costs a few cycles on average (with our tidy numbers, about $0.15 \times 40 = 6$ ). ARM's clever move: don't fork at all. Stamp a little "only if…" tag on each instruction. The line stays straight, nothing is ever fetched down a wrong path, and skipped instructions just do nothing for one tick. That's a bargain as long as you're only skipping a handful — up to about six instructions. Skip more than that and you'd have been better off with one clean jump, which is exactly why the 64-bit ARMv8, riding on excellent modern predictors, threw the trick away and trusts the branch predictor instead. The lesson in one breath: forks are expensive because they force a guess; tagging instructions avoids the guess entirely — a great deal for short, unpredictable conditionals, and a poor one for long blocks.

Recall Quick self-test

Why can a predicated (conditional) instruction never cause a pipeline flush? ::: Because the instruction that follows it in memory is always the next one fetched — there is no fork, so the fetch stage never guesses, so there is nothing to un-guess. What three quantities appear in the honest branch-cost formula? ::: The correct-prediction toll $C_{ok}$ , the mispredict probability $P_{mispredict}$ , and the flush penalty $C_{flush}$ . Is $C_{flush} \approx 40$ a real spec? ::: No — it is a pessimistic teaching figure; real Cortex-A mispredict penalties are commonly ~15–20 cycles, shaped by BTBs, decode/dispatch timing and out-of-order recovery. Why did ARMv8 remove conditional execution? ::: Modern branch predictors exceed 95% accuracy (so $C_{branch}$ is tiny) and predication complicates out-of-order execution; for anything but the shortest blocks, a clean branch now wins. Are $C_{cond}$ and $C_{NOP}$ the same thing? ::: Yes — both equal 1 cycle; $C_{cond}$ names the cost of any predicated instruction, $C_{NOP}$ just stresses the cost of one whose condition was false.

See also: 5.1.07-ARM-instruction-encoding (where the 4-bit condition field lives) · 5.2.03-pipelining (stages in depth) · 6.3.02-cache-memory (why fewer/smaller instructions help) · 7.1.05-power-optimization (fewer flushes = less wasted energy) · 5.1.10-x86-architecture (the variable-length contrast).