Von Neumann architecture — components, bottleneck: Visual walkthrough

We are going to answer one question: how many instructions per second can this machine actually run, given that everything squeezes through one hallway?

Step 1 — Draw the machine as two rooms and one door

WHAT. A Von Neumann machine has exactly two places that matter for our story: the CPU (the thinker) and memory (the store). Between them is a single shared path — the bus (see Buses (Address, Data, Control)). Everything the CPU ever needs — every instruction, every piece of data — must walk through that one door.

WHY draw it this way? Because the bottleneck is not a property of the CPU or of memory alone. It is a property of the door between them. If we don't draw the door, we can't see the jam. So the very first move is to make the shared path visible.

PICTURE. The CPU on the left, memory on the right, one bus connecting them. Notice there is only one wire — not two. That single wire is the whole story.

Figure — Von Neumann architecture — components, bottleneck

Step 2 — Define the smallest unit of traffic: one word per bus cycle

WHAT. A word is one chunk of bits the bus can carry in one go (say, 64 bits). A bus cycle is the time it takes the bus to carry exactly one word from memory to the CPU (or back). So the bus's rule is brutally simple:

$how much moves 1 word per how long it takes 1 bus cycle$

WHY these two words first? Because throughput is stuff over time. We cannot talk about "instructions per second" until we agree what one unit of "stuff" (a word) and one tick of "time" (a bus cycle) are. Everything downstream is counted in these two units.

PICTURE. A single word — a little packet — sliding down the bus during one cycle. The door fits exactly one packet at a time. A second packet must wait.

Step 3 — Count what a single instruction actually costs in trips

WHAT. Running one instruction is not one trip through the door. Watch the Fetch-Decode-Execute Cycle:

1 trip to fetch the instruction itself — the CPU must first bring the instruction across the bus before it can even know what to do.
Then $d$ more trips for data — the operands it reads and the results it writes. We call this number $d$ the average data accesses per instruction.

Add them up. The number of words moved per instruction is:

$T = the instruction fetch — always 1 + its data trips, on average d$

WHY the "+1" is not optional. Here is the deep point people miss: in Von Neumann, the instruction and its data share the same door. The instruction is not free — fetching it is a bus trip too, competing with the data. That inescapable "+1" is the fingerprint of a shared memory. (In Harvard Architecture the instruction rides a separate bus, so it would not add to this count — that is exactly what makes Harvard different.)

PICTURE. One instruction unfolding into its trips: first a lavender trip (fetch), then $d$ coral trips (data). Here $d = 2$ , so $T = 1 + 2 = 3$ packets march single-file through the one door.

Step 4 — Turn "trips per instruction" into "instructions per second"

WHAT. The bus supplies words at a fixed rate. Call that rate $B$ = words per second the bus can deliver (if the bus does $1 0^{9}$ cycles/sec and carries one word each, then $B = 1 0^{9}$ ). Every second, the bus hands over $B$ words. Each instruction eats $T$ words. So the number of instructions you can finish per second is just:

$IPS_{m a x} = \frac{B words the bus gives per second}{words one instruction eats T} = \frac{B}{1 + d}$

WHY divide? This is the same logic as "if a bakery makes $B$ eggs a day and each cake needs $T$ eggs, you can bake $B / T$ cakes." Total supply ÷ cost-per-item = number of items. Nothing fancier — throughput is always a division.

PICTURE. A one-second-wide bar filled with $B$ word-slots. We slice it into chunks of size $T$ . The number of chunks that fit is the max instruction rate. Fewer, fatter chunks (bigger $d$ ) → fewer instructions.

Step 5 — Plug in numbers so the wall becomes real

WHAT. Let the bus deliver $B = 2 \times 1 0^{9}$ words/s, and let each instruction average $d = 2$ data accesses. Then $T = 1 + 2 = 3$ and:

$IPS_{m a x} = \frac{2 \times 1 0 ^{9}}{3} \approx 6.67 \times 1 0^{8} instructions/s = 0.667 GIPS$

WHY this specific example matters. Imagine the CPU's thinking circuits could run at 10 GHz. Doesn't matter — the door caps it at 0.667 GIPS. The wall is the bus, not the CPU. That gap between "how fast the CPU could think" and "how fast the door lets it" is the Von Neumann bottleneck made numeric.

PICTURE. Two bars side by side: a tall lavender bar (CPU could do 10 GIPS) and a short coral bar (bus allows 0.667 GIPS). The coral bar is the ceiling. The wasted lavender on top is the stalled CPU.

Step 6 — The same story from the CPU's side: utilisation

WHAT. Now stand inside the CPU. For one instruction it spends:

$t_{c}$ = compute time — time actually doing useful logic (the ALU adds, compares...).
$t_{m}$ = memory-wait time — time frozen, staring down the hallway, waiting for words to arrive.

The fraction of time doing something useful — the utilisation — is useful time over total time:

$U = \frac{t _{c} working}{working t _{c} + waiting t _{m}}$

WHY a second formula? IPS looked at the bus. Utilisation looks at the CPU's idleness. They are two views of one jam: the bus being the limiter (Step 4) is the same event as the CPU sitting idle here. Seeing both is what makes the concept click.

PICTURE. A time-line for one instruction: a small mint slice of "compute" followed by a long butter slice of "waiting". $U$ is the mint fraction of the whole bar.

Step 7 — The limiting case: what happens as memory falls behind

WHAT. History's trend: CPUs got fast much quicker than memory. So $t_{m}$ grows relative to $t_{c}$ . Push it to the extreme — let $t_{m}$ get enormous compared to $t_{c}$ :

$lim_{t_{m} \to \infty} U = lim_{t_{m} \to \infty} \frac{t _{c}}{t _{c} + t _{m}} = 0$

WHY take a limit? Because the bottleneck isn't "the CPU wastes some time" — it's a trend that gets worse without bound. A limit is the tool that answers "where is this heading?". And it heads to zero: eventually the CPU does almost nothing but wait. This is the memory wall.

Two edge cases pin the range so you never meet a surprise:

$t_{m} = 0$ (memory instant / everything in registers): $U = \frac{t _{c}}{t _{c} + 0} = 1 = 100%$ — no bottleneck at all.
$d = 0$ (an instruction with no data trips at all, e.g. a NOP): $T = 1 + 0 = 1$ , so $IPS_{m a x} = B$ — the bus is used purely for fetching. Even here it is still one door; you just aren't asking data to compete this instant.

PICTURE. A curve of $U$ against the ratio $t_{m} / t_{c}$ , sliding from $U = 1$ at the left (fast memory) down toward $U = 0$ on the right (slow memory). The shaded region under the falling curve is "the bottleneck growing".

The one-picture summary

Everything above, compressed: one door, $T = 1 + d$ trips per instruction, bus rate $B$ giving $IPS_{m a x} = B / (1 + d)$ , and the CPU idling for the fraction $\frac{t _{m}}{t _{c} + t _{m}}$ of every instruction.

Recall Feynman retelling — the whole walkthrough in plain words

Picture a chef with one narrow hallway to a single pantry that holds both recipe cards and ingredients. To do one dish the chef must run the hallway once for the recipe card, then once more for each ingredient — that's the " $1 + d$ " trips. The hallway lets only one person through per moment, and it can push through, say, $B$ items each second; so the number of dishes per second is just "items the hallway supplies" ÷ "items each dish needs" $= B / (1 + d)$ . From the chef's own view, he chops for a tiny moment ( $t_{c}$ ) then stands waiting in the hallway for a long moment ( $t_{m}$ ); the slice of time he's actually cooking is $t_{c} / (t_{c} + t_{m})$ . As pantries got slower relative to chefs, that slice shrank toward nothing — the chef spends the whole day running, not cooking. That endless running through one hallway is the Von Neumann bottleneck. Give him a small shelf of favourites by the cutting board (a cache) and most trips vanish — but the pantry is still one room with one hallway underneath.