4.1.1 · D5Computer Architecture (Deep)

Question bank — Von Neumann architecture — components, bottleneck

2,304 words10 min readBack to topic

Building the two formulas (so the symbols mean something)

Figure — Von Neumann architecture — components, bottleneck
Figure 1 — CPU and MEMORY connected by one red shared bus; every instruction needs fetch data transfers through it, giving .

Figure — Von Neumann architecture — components, bottleneck
Figure 2 — one instruction's timeline: red compute slice over the whole bar is the utilisation ; the empty tail is idle waiting.


True or false — justify

Von Neumann architecture is defined by "having a CPU and a memory".
False — every computer has those. The defining choice is that instructions and data live in one memory reached over one shared bus. The word is shared, not exists.
If memory were made as fast as the CPU, the Von Neumann bottleneck would disappear.
False — the bottleneck is structural. Even at equal speed, a single shared bus forces instruction transfers and data transfers to take turns (serialize). Speed only sharpens a problem that sharing already creates.
Harvard Architecture avoids the bottleneck by using faster memory.
False — it uses separate memories and buses for code and data, so a fetch and a data access can happen simultaneously. The fix is a second channel, not faster silicon.
A cache converts a Von Neumann machine into a Harvard machine.
False — split L1-I / L1-D caches ("modified Harvard") only apply at the top level; main memory stays unified. The machine is still fundamentally Von Neumann; the cache hides the bottleneck, it does not remove it.
The stored-program idea means "a program is stored in memory".
Partly (right in spirit, wording too shallow) — the sharp point is that an instruction is itself just a number, indistinguishable in form from data, so it lives in the same store — see Stored Program Concept. That sameness, not mere "storage", is what makes code loadable and rewritable.
Because instructions and data look identical in memory, a program could in principle overwrite its own instructions.
True — nothing about the bits distinguishes them; a value written to the wrong address can land on code. This is a direct consequence of the shared, undifferentiated store (and the root of self-modifying code and many security exploits).
The Program Counter (PC) is part of memory.
False — the PC is a register inside the Control Unit (see CPU Registers). It holds an address into memory, but it is not itself stored in main memory.
Fetch and Execute happen at the same instant to save time.
False — the Fetch-Decode-Execute Cycle runs them in sequence for one instruction. The whole point of the bottleneck is that the one bus cannot serve both a fetch and a data access simultaneously.
Higher clock speed always raises the instructions-per-second of a Von Neumann machine.
False — once is the wall, a faster clock just makes the CPU idle more while waiting. Beyond that point the bus, not the clock, is the limit.
The bus-limited formula still describes a modern pipelined, out-of-order CPU exactly.
Partly — it captures the bus ceiling correctly, but real CPUs use pipelining, out-of-order execution, and non-blocking caches to overlap waiting with useful work, so measured IPS can approach that ceiling far better than the simple "wait then compute" picture suggests. The formula is the limit, not the everyday behaviour.

Spot the error

"The five components are Memory, CPU, ALU, I/O, and Bus."
Error: CPU double-counts. The five classic parts are Memory, Control Unit, ALU, I/O, and Bus — the CU + ALU + registers together form the CPU.
"After Fetch, the PC (Program Counter) is incremented only if the instruction is not a jump."
Error: the PC is incremented always, right after fetch, so the default "next" is ready. A jump then simply overwrites the PC — see Fetch-Decode-Execute Cycle.
", because you divide bus throughput by the data accesses."
Error: it is . The +1 is the instruction fetch itself, which also travels the shared bus. Forgetting it ignores the very traffic that causes the bottleneck.
"Decode is where the machine's state changes."
Error: Decode only interprets the opcode bits into control signals. Execute is the step that meaningfully changes state (writes a register, memory, or the PC).
"With the bottleneck vanishes, so IPS is unlimited."
Error: even at , — still capped by how fast the bus supplies instructions. Fewer data accesses ease it; they never remove it.
"Utilisation is the fraction of useful work."
Error: it is inverted (see Figure 2). Useful work is compute time over total time, ; the quoted ratio is the idle fraction.

Why questions

Why does storing instructions as numbers make computers reprogrammable "on the fly"?
Because loading a new program becomes just writing different numbers into memory, a fast memory operation — versus rewiring hardware, which took days on pre-1945 machines.
Why is the bottleneck called a bottleneck rather than just "slow memory"?
Because the single shared bus is a narrowing through which all traffic (code + data) must squeeze one item at a time — a geometric constriction, like the neck of a bottle, independent of raw speed.
Why increment the PC (Program Counter) before executing, not after?
So the "default next instruction" address is already correct for the common case; branch/jump instructions then only need to overwrite the PC, keeping the control logic uniform.
Why does a cache raise utilisation without touching the bus?
It keeps hot instructions/data in fast storage next to the CPU, so most accesses never travel the shared bus — cutting average access time and the CPU's waiting (see Memory Hierarchy and Caching).
Why does Harvard architecture appear in DSP chips and microcontrollers but pure Von Neumann dominates general PCs?
Harvard's parallel code/data fetch suits fixed, streaming workloads; Von Neumann's single unified memory is simpler and more flexible for general programs where code/data proportions vary — a design trade, not a ranking.
Why do split L1 caches count as "modified Harvard" but not full Harvard?
They give separate fast code and data paths at the top level (Harvard-like parallelism), yet both draw from one unified main memory below — so the Von Neumann core remains.
Why must the CPU pull an instruction into the IR (Instruction Register) before decoding, rather than decoding it in place?
Memory is too far and slow to drive the control logic directly; the CPU decodes bits held inside itself in the Instruction Register (IR), where control signals can be generated at CPU speed.
Why can pipelining and non-blocking caches make the CPU look like it beats the simple bottleneck model?
Because they let the CPU do useful work during the wait (issuing later instructions, servicing other accesses), so idle time overlaps with compute time instead of stacking after it — raising real toward the ceiling without changing that ceiling.

Edge cases

If an instruction needs zero data accesses (), is there still bus traffic?
Yes — the instruction fetch itself uses the bus, so and . The floor is one transfer per instruction, never zero.
What is the utilisation when the CPU never has to wait for memory ()?
— the ideal, no-bottleneck limit. Real systems approach it only as caches make effective .
As memory grows relatively slower and slower (), what happens to ?
— the CPU spends essentially all its time waiting. This limiting behaviour is the bottleneck in its extreme form.
If cache hit rate reaches 100%, does the Von Neumann bottleneck cease to exist?
No — it becomes invisible in practice, not gone. The unified main memory and shared bus still exist; any miss, cold start, or cache-bypassing access exposes the same structural limit.
Can a single-instruction program ever stall on the Von Neumann bottleneck?
Yes — even one instruction that reads and writes data must serialize its fetch and its data transfers over the one bus. The bottleneck is per-access contention, not a large-program-only effect.
What happens to as data-heavy instructions push very high?
as — instruction rate collapses because each instruction monopolizes the bus for many data words, starving the fetch of the next one.
Does out-of-order execution remove the shared-bus limit, or just hide the waiting?
It hides it — reordering fills idle slots with independent work, but every fetch and data access still crosses the same one bus, so remains the hard ceiling.

Recall One-line summary of the whole trap set

Almost every trap reduces to one confusion: mistaking shared single channel (the real definition) for slow memory (only an aggravating factor). Test any claim by asking "does this depend on sharing, or on speed?"

Related: Buses (Address, Data, Control) · Instruction Set Architecture · Stored Program Concept · parent topic