Exercises — Von Neumann architecture — components, bottleneck

Defining choice: instructions and data live in one shared memory, reached over one shared bus. Contrast: Harvard Architecture, which gives instructions and data separate memories and separate buses, so it can fetch an instruction and access data at the same time. The distinguishing word is shared, not exists — every computer has a CPU and memory.

Memory, Control Unit (CU), Arithmetic Logic Unit (ALU), Input/Output (I/O), Bus. The CU + ALU + registers together form the CPU. (A register is a tiny fast storage slot inside the CPU — see the definition box above and CPU Registers.) Mnemonic: My Computer Always Is Busy → M-C-A-I-B.

The "$1$" is the instruction fetch — the one bus trip you always make to bring the instruction into the CPU, before touching any data. (Recall from the box above that the whole denominator is $T=1+d$, the total bus words per instruction.) $d=0$ means an instruction that touches no data at all (e.g. a NOP or a register-only operation like "clear a flag"). Then $T=1+0=1$, so ${\text{IPS}}_{\max }=B$: the bus is used only for fetching instructions and nothing slows it further.

What we do: count words per instruction ($T$), then divide the bus throughput by it. $T=1+d=1+3=4\text{ words/instruction}$ Why: 1 word to fetch the instruction + 3 words of data. ${\text{IPS}}_{\max }=\frac{B}{T}=\frac{4\times 10^9}{4}=1\times 10^9=1\text{ GIPS}$ (Recall GIPS = giga-instructions/sec = $10^9$ instructions/sec, defined in the tools box.) Even a CPU that could "think" faster is capped at 1 billion instructions/second by the bus.

Why this formula: $U$ is useful time ÷ total time. Total time per instruction $=t_c+t_m$. $U=\frac{t_c}{t_c+t_m}=\frac{2}{2+6}=\frac{2}{8}=0.25=25\%$ The CPU is doing useful work only 25% of the time, so it is idle 75% of the time — waiting at the door.

Why a weighted average: each access is either a hit or a miss; weight each time by how often it happens. $\bar{t}=0.95(1)+0.05(20)=0.95+1.00=1.95\text{ ns}$ We cut the effective wait from a possible 20 ns down to 1.95 ns without touching the bus — this is Memory Hierarchy and Caching hiding the bottleneck, not removing it.

Why compute both: a bigger $B$ does not automatically win — the effective divisor $1+d$ matters too. ${\text{IPS}}_A=\frac{8\times 10^9}{1+3}=\frac{8\times 10^9}{4}=2\times 10^9$ ${\text{IPS}}_B=\frac{6\times 10^9}{1+1}=\frac{6\times 10^9}{2}=3\times 10^9$ B wins, despite its slower bus, because its instructions are "leaner" ($d=1$). $\text{ratio}=\frac{3\times 10^9}{2\times 10^9}=1.5$ B is 1.5× faster in instructions/sec. Lesson: reducing $d$ (fewer memory operands per instruction) can beat raw bus speed — this is part of why a lean Instruction Set Architecture matters.

Why an average $T$: different instructions cost different numbers of bus trips; weight each cost by how often it runs. First average $d$: $\bar{d}=0.6(1)+0.4(4)=0.6+1.6=2.0$ Now be careful going from $\bar{d}$ to $\bar{T}$. Each instruction's cost is $T=1+d$, so the average cost is $\bar{T}=0.6(1+1)+0.4(1+4).$ The constant "$1$" appears in every term, so it factors straight out of the weighted sum (the weights $0.6+0.4=1$): $\bar{T}=\underset{=1}{\underbrace{(0.6+0.4)}}\cdot 1+[0.6(1)+0.4(4)]=1+\bar{d}.$ That is why $⟨1+d⟩=1+⟨d⟩$ — the fixed fetch cost is not being averaged, only the variable data cost is. So: $\bar{T}=1+\bar{d}=1+2.0=3.0\text{ words/instruction}$ ${\text{IPS}}_{\max }=\frac{B}{\bar{T}}=\frac{10^{10}}{3.0}\approx 3.33\times 10^9\text{ IPS}$ The blue blocks in the figure are the data traffic; the taller the stacked bar, the more bus trips that instruction type costs. Weighting those heights by how often each type runs gives the red dashed mean, $\bar{T}=3.0$.

Original: $U_1=\frac{2}{2+8}=\frac{2}{10}=0.20=20\%$. After halving $t_m$: $U_2=\frac{2}{2+4}=\frac{2}{6}=0.333\dots =33.3\%$. Doubling would predict $40\%$. Actual is $33.3\%$ — not double. Why: $U$ is not linear in $t_m$; $t_c$ sits in the denominator too, so halving $t_m$ helps less than proportionally. The claim confuses "halve the wait" with "double the useful fraction."

Baseline: ${\text{IPS}}_0=\frac{B}{1+3}=\frac{B}{4}$. Option (a) — double $B$: ${\text{IPS}}_a=\frac{2B}{1+3}=\frac{2B}{4}=\frac{B}{2},\text{gain}=\frac{B/2}{B/4}=2\times$ Option (b) — drop $d$ to 1: ${\text{IPS}}_b=\frac{B}{1+1}=\frac{B}{2},\text{gain}=\frac{B/2}{B/4}=2\times$ Both give exactly $2\times$. Why the coincidence (in words): IPS is a fraction, $\frac{\text{numerator }B}{\text{divisor }1+d}$. There are two ways to double any fraction: double its top or halve its bottom — both land on the same value. Option (a) doubles the top ($B\to 2B$). Option (b) halves the bottom: the divisor goes from $1+3=4$ down to $1+1=2$, which is exactly half. Since "double the top" and "halve the bottom" always produce the same doubled fraction, the two very different engineering changes must give the identical $2\times$ result. The design lesson: reducing memory traffic per instruction can be just as powerful as buying faster hardware — and it is often cheaper. This is the philosophy behind register-heavy Instruction Set Architecture and CPU Registers.

Why Von Neumann stalls: one bus carries both the instruction fetch (1 word) and the data ($d$ words), so they must serialize — total $1+d$ trips on one channel. Harvard's move: put instructions on their own bus $B_I$ and data on their own bus $B_D$. Now the two streams flow in parallel instead of taking turns. The instruction rate is limited by whichever bus finishes last: ${\text{IPS}}_{\text{Harvard}}=\min \left(\frac{B_I}{1},\frac{B_D}{d}\right),d\ge 1.$ Domain note ($d=0$): the term $B_D/d$ divides by $d$, so it is only meaningful when $d\ge 1$. If $d=0$ the data bus carries nothing, so it never limits anything; the rate is then set purely by the instruction bus, ${\text{IPS}}_{\text{Harvard}}=B_I$. (Formally, take $\min$ only over the buses that are actually used.) For every problem on this page $d\ge 1$, so we use the full $\min$ expression. With equal-speed buses $B_I=B_D=B$ and $d=3$: $\min (B,B/3)=B/3$, versus Von Neumann's $B/4$ — since $B/3>B/4$, Harvard is faster. Same memory speed, higher rate — because there is no contention, purely by splitting the channel. This connects to Buses (Address, Data, Control) and the Fetch-Decode-Execute Cycle overlapping fetch with data access.

Plan P (Von Neumann): ${\text{IPS}}_P=\frac{B}{1+d}=\frac{5\times 10^9}{1+4}=\frac{5\times 10^9}{5}=1\times 10^9\text{ IPS}$ Plan Q (Harvard), with $d=4\ge 1$ so the full $\min$ applies: ${\text{IPS}}_Q=\min \left(\frac{B_I}{1},\frac{B_D}{d}\right)=\min \left(\frac{2\times 10^9}{1},\frac{3\times 10^9}{4}\right)=\min (2\times 10^9,0.75\times 10^9)=0.75\times 10^9\text{ IPS}$ Recommendation: Plan P ($1$ GIPS $>$ $0.75$ GIPS). Why Q lost: with $d=4$, the data bus is the bottleneck — $B_D/4=0.75$ GIPS drags the whole machine down. Splitting the channel only helps if you split the throughput sensibly for the workload; here the data-heavy program starves the narrow data bus. Non-performance reason to still favour Plan P: the Stored Program Concept — a single unified memory lets programs be treated as data (self-modifying code, JIT compilation, simple loaders, one address space). Harvard's split memories make treating instructions as writable data awkward. Simplicity and flexibility, not just speed.

Domain: we assume $d\ge 1$, so that $B_D/d$ is well-defined and the data bus actually carries traffic. (The $d=0$ case is handled in L4.2: there Harvard $=B_I=B$ while Von Neumann $=B/1=B$, a tie — so the strict inequality below needs $d\ge 1$.) Von Neumann rate: ${\text{IPS}}_{VN}=\frac{B}{1+d}.$ Harvard rate (equal buses $B$): ${\text{IPS}}_H=\min \left(\frac{B}{1},\frac{B}{d}\right).$ For $d\ge 1$ we have $d\ge 1$, so $\frac{B}{d}\le \frac{B}{1}$, which means the data bus is the slower one and ${\text{IPS}}_H=\frac{B}{d}.$ Take the ratio (Harvard over Von Neumann): $\frac{{\text{IPS}}_H}{{\text{IPS}}_{VN}}=\frac{B/d}{B/(1+d)}=\frac{B}{d}\cdot \frac{1+d}{B}=\frac{1+d}{d}=1+\frac{1}{d}.$ Conclusion: since $\frac{1}{d}>0$ for every $d\ge 1$, the ratio is always strictly greater than 1 — so ${\text{IPS}}_H>{\text{IPS}}_{VN}$, i.e. Von Neumann never beats Harvard here. The exact advantage is a factor $1+\frac{1}{d}$. Why the advantage shrinks as $d$ grows: as instructions carry more and more data, the data bus dominates both designs, so the "free" parallel instruction bus becomes a smaller slice of total work. Check the extremes: at $d=1$ the ratio is $1+\frac{1}{1}=2$ (Harvard doubles the rate); at $d=10$ it is $1+\frac{1}{10}=1.1$ (only 10% faster). Harvard's edge is largest for lean, data-light instruction streams.

Why is the divisor $1+d$ and not $d$? ::: You always spend one bus trip fetching the instruction before touching any data. When can a slower bus win on IPS? ::: When its instructions are leaner (smaller $d$), because the effective divisor $1+d$ is smaller. Why doesn't halving $t_m$ double $U$? ::: $t_c$ also sits in the denominator, so $U$ is non-linear in $t_m$. Harvard vs Von Neumann speed ratio at equal buses? ::: $1+\frac{1}{d}$ — always above 1, shrinking as $d$ grows. Why keep Von Neumann despite Harvard being faster? ::: The stored-program idea (code as data) needs one unified writable memory.