Visual walkthrough — Harvard architecture — separate instruction - data memory

This is a companion to the parent topic — a slow, visual re-derivation.

---

Step 1 — What a "cycle" even is

WHAT. We agree on our unit of time: 1 cycle = 1 tick = one trip across a bus.

WHY. Before we can count how long a program takes, we need something to count in. Seconds vary chip to chip; cycles are the honest, chip-independent unit. Every claim on this page is "how many ticks?".

PICTURE. Look at the row of tick-marks below. Each box is one cycle. A wire (a "bus") can move exactly one item — a recipe 📜 or an ingredient 🥕 — during one box.

---

Step 2 — The two jobs every instruction may need

WHAT. We split all work into exactly two kinds of memory trips.

WHY. The whole Harvard idea lives in the gap between these two. If we don't separate "getting the recipe" from "getting the ingredient", we can't talk about doing them in parallel.

PICTURE. Two colored trays: burnt-orange = fetch trips, teal = data trips. Notice not every instruction spawns a teal trip.

---

Step 3 — Naming the workload with symbols

We earn every letter before using it.

WHAT. We captured "how much work" in two numbers.

WHY. We want a formula, not one specific answer. $N$ and $f$ let the same picture describe any program — from pure arithmetic ($f=0$) to memory-hammering ($f=1$).

PICTURE. A bar of $N$ instructions; the teal slice is the $fN$ that need an ingredient.

Recall Check yourself

If $N=1000$ and $f=0.4$, how many data-touching instructions? ::: $fN=0.4\times 1000=400$.

---

Step 4 — Counting cycles the von Neumann way (one bus)

WHAT. Because the bus is shared, the fetch trip and the data trip cannot share a tick. They must go in different boxes.

WHY they ADD. Two jobs forced onto one wire, one-at-a-time, means their costs stack up:

• All $N$ fetches → $N$ cycles.
• All $fN$ data trips → another $fN$ cycles, because the bus was busy fetching and couldn't do them at the same time.

$T_{\text{vN}}=\underset{\text{fetches}}{\underbrace{N}}+\underset{\text{data trips}}{\underbrace{fN}}=N(1+f)$

• $T_{\text{vN}}$ — total ticks on the single-bus design.
• $N$ — the unavoidable fetches.
• $fN$ — the extra ticks stolen because data had to wait its turn on the same bus. This waiting is the Von Neumann bottleneck.

PICTURE. Watch the single timeline: orange, orange, then a data instruction forces an extra teal box before moving on. The timeline gets longer every time teal appears.

---

Step 5 — Counting cycles the Harvard way (two buses)

WHAT. The data trip no longer needs its own box. It slides into the same tick as the fetch, on the other wire.

WHY they OVERLAP (max, not sum). With two wires the two jobs run side-by-side. The time for a tick is the longer of the two jobs, not their sum. Since both are 1 cycle: $\text{cost per instruction}=\max (1,1)=1(\text{not }1+1)$ So all $N$ instructions finish in: $T_{\text{H}}=N$

• $T_{\text{H}}=N$ — the $fN$ data trips cost zero extra, because they hid underneath fetches already happening.

PICTURE. Two parallel tracks. The teal data trip sits directly below an orange fetch in the same box — no new box appears. The timeline stays length $N$.

---

Step 6 — Divide the two timelines: the speed-up

WHAT. We stack the two timelines from Steps 4 and 5 and take their ratio.

WHY divide? A difference ($T_{\text{vN}}-T_{\text{H}}=fN$) depends on program size $N$ and isn't portable. A ratio cancels $N$ and gives a clean, size-free verdict.

$S=\frac{T_{\text{vN}}}{T_{\text{H}}}=\frac{N(1+f)}{N}=1+f$

• Numerator $N(1+f)$ — von Neumann's stacked cycles (Step 4).
• Denominator $N$ — Harvard's overlapped cycles (Step 5).
• The $N$'s cancel → the answer depends only on $f$, the memory pressure.

PICTURE. Two bars side by side; the orange (vN) bar is longer by exactly its teal $fN$ tail. The ratio of lengths is the number in the box.

---

Step 7 — The two edge cases (never leave a scenario unshown)

Case $f=0$ — pure register arithmetic, no data trips. $S=1+0=1$

• WHAT: no teal trips exist.
• WHY: the data bus sits idle — there's nothing to overlap. Harvard's second wire does nothing, so no speed-up. The extra hardware was wasted for this workload.

Case $f=1$ — every instruction touches data. $S=1+1=2$

• WHAT: every tick has both an orange and a teal job.
• WHY: von Neumann needs $2N$ boxes (fetch + data, stacked, every time); Harvard needs $N$ (fully overlapped). Exactly $2\times$. This is the best Harvard can ever do under our 1-cycle model.

In between: real programs sit at $f\approx 0.3\text{–}0.4$, so $S\approx 1.3\text{–}1.4$. The benefit scales with memory pressure — it is not a fixed magic number.

PICTURE. The line $S=1+f$ plotted from $f=0$ to $f=1$: starts at 1, ends at 2, straight line. The two endpoints are marked.

---

The one-picture summary

Everything above, compressed: two timelines, the overlap that hides the data trip, and the ratio that becomes $1+f$.

Recall Feynman: tell the whole walkthrough in plain words

Picture a chef doing $N$ jobs. Every job needs a recipe (fetch); some fraction $f$ of jobs also need an ingredient (data). In the one-door kitchen (von Neumann) the chef fetches the recipe, then walks back to the same door for the ingredient — two trips through one door, so the ingredient jobs make the day longer: $N+fN$ trips. In the two-door kitchen (Harvard), a helper brings the ingredient through a second door at the same moment the recipe arrives — the ingredient trip hides under the recipe trip and costs no extra time: still just $N$ ticks. Divide the long day by the short day and the $N$ cancels, leaving $1+f$. Test the ends: if no job needs ingredients ($f=0$) the second door is useless and you save nothing; if every job needs one ($f=1$) you finish twice as fast. That single line — $S=1+f$ — is the entire story.

---

Connections

• Von Neumann architecture — the one-bus design of Steps 4 and 6.
• Von Neumann bottleneck — the "waiting turn" tail in Step 4.
• CPU instruction cycle — the fetch step we overlapped with data.
• Cache memory — split I/D caches make this overlap real in modern chips.
• Pipelining — overlapping stages, same spirit as overlapping buses.
• Memory bus and bandwidth — two buses = the parallel wires of Step 5.
• Microcontrollers (AVR PIC ARM Cortex-M) — chips that live by this speed-up.