6.2.1 · D5GPU Architecture
Question bank — GPU vs CPU design philosophy
This is a self-test bank for GPU vs CPU design philosophy. Every line below is a trap: a claim that sounds right until you reason carefully. Read the left side, answer out loud in one or two sentences, then reveal.
True or false — justify
A GPU core is faster than a CPU core
False. Each individual GPU core is slower per instruction (no branch prediction, no out-of-order, lower clock); the GPU only wins by running thousands of these slow cores at once.
A GPU always beats a CPU
False. Only on throughput work with many independent tasks. On a single sequential task (one thread of dependent steps) the CPU's low latency wins — see the binary-search trap below.
CPUs cannot do parallel work
False. CPUs have multiple cores (8–16) and do run threads in parallel; they simply devote most transistors to making each thread fast rather than to having many threads.
More cores always means more throughput
False. Throughput only rises with cores if you have enough independent tasks to feed them and enough memory bandwidth; idle cores add nothing (this is the term in the GPU model).
A larger cache would make a GPU faster
False in general. Cache is expensive per core; at thousands of cores, giving each a big cache would leave no transistor budget for arithmetic. The GPU hides latency by thread switching instead of caching.
Branch prediction would help a GPU a lot
False mostly. The GPU hides latency by having other warps ready, so it doesn't need speculation the way a serial CPU does; adding predictors would waste transistors better spent on cores.
Higher clock frequency is the main reason CPUs feel "fast"
Partly false. The clock helps, but the dominant reason single tasks finish fast is high IPC from caches, branch prediction, and out-of-order execution — clocks have barely risen in years.
The GPU's advantage comes from a higher clock speed
False. GPU clocks are usually lower than CPU clocks. The advantage is the number of parallel cores, not clock speed.
If a workload has data dependencies between steps, a GPU still helps
False. Dependent steps must run in order; there is no parallelism to exploit, so the GPU's slower cores just make it worse.
Occupancy of 100% is always achievable and always ideal
False. Register and shared-memory limits cap resident warps, so occupancy is typically 0.5–0.9; and once latency is fully hidden, extra occupancy gives no further speedup.
Spot the error
"GPUs are faster because they have more transistors."
Wrong reason. The A100 has more transistors, but the point is how they are spent — a far larger fraction goes to arithmetic units rather than to caches and control logic. Spending is the story, not raw count.
"The CPU wastes most of its die on caches, which is bad design."
Wrong judgement. Those caches exist to cut memory latency (100 ns DRAM → 1–4 ns cache hit) for sequential code — excellent design for the latency question the CPU answers.
"A warp is 32 separate programs running independently."
Wrong. A warp is 32 threads all executing the same instruction in lockstep (SIMT). They share the instruction stream; only their data differs.
"When a warp stalls on memory, the whole GPU stalls."
Wrong. The scheduler instantly switches to another ready warp, so the arithmetic units keep working. That switching is exactly how the GPU hides DRAM latency without caches.
"Speedup for the blur example is unlimited if we add more cores."
Wrong. Speedup is capped by memory bandwidth, the number of independent pixels, and eventually by any serial portion — see Amdahl's law.
" being higher on a GPU means the GPU is a bad design."
Wrong. When , per-instruction latency is hidden by parallelism, so a higher barely affects total time — the trade-off is intentional.
"Both CPU and GPU follow the von Neumann model, so their performance is basically the same."
Wrong. Both share the von Neumann fetch-execute idea, but their microarchitecture (how many cores, how latency is hidden) makes them opposite in behaviour.
Why questions
Why do CPUs use out-of-order execution but GPUs use in-order?
The CPU has few threads, so it must extract parallelism within one instruction stream (ILP) via reordering. The GPU has thousands of threads, so it finds parallelism across threads and never needs to reorder within one.
Why does a GPU need many resident warps per SM?
To hide the hundreds of cycles of DRAM latency: while dozens of warps wait on memory, others keep the arithmetic units busy. Too few warps and the cores would sit idle.
Why do branches hurt GPUs more than CPUs?
A warp runs one instruction for all 32 threads; if threads take different branches, the hardware must serialize the paths (branch divergence), doing each side while masking the others — wasted work the SIMT model can't avoid.
Why can't a GPU just add a big branch predictor and beat the CPU at everything?
That transistor budget is fixed (). Spending it on prediction means fewer cores, killing the throughput advantage that is the GPU's whole reason to exist.
Why is the restaurant analogy (chef vs cafeteria) actually accurate?
The chef (CPU) minimizes time per order — latency; the cafeteria (GPU) maximizes orders per hour — throughput. Each is worse at the other's job by design, which is exactly the CPU/GPU split.
Why does maximizing (processor count) help even if goes up?
Because : for large , dividing by a bigger shrinks time faster than a modest rise in grows it — so accepting slower cores to get many more of them is a net win.
Edge cases
What happens on a GPU when ?
The GPU is worse than a CPU: only one slow core does useful work, the other thousands are idle, and there is no latency to hide. This is the degenerate serial case.
What happens when (not )?
The GPU's throughput advantage nearly vanishes; with no surplus threads to hide memory stalls, cores idle and the slower per-instruction time starts to dominate.
What happens if every thread in a warp takes a different branch?
Worst case: all 32 paths serialize, so the warp runs up to 32× slower for that region — the SIMT model turns a 32-wide unit into effectively a 1-wide one.
What if the workload fits entirely in the CPU's L3 cache but is huge on the GPU?
The CPU may win: cache hits give it ~1–4 ns memory access while the GPU pays hundreds of ns to DRAM. Small, cache-friendly, latency-bound work favours the CPU.
Occupancy = 0 — what does the GPU model predict?
: no cores are actively computing, so nothing finishes. It flags that a GPU with zero active warps does zero useful work regardless of core count.
What if a "parallel" task secretly has one serial bottleneck (a global sum step)?
That serial fraction caps total speedup no matter how many cores you add — the formal statement is Amdahl's law; the parallel model only holds for the fully independent part.
Zero-latency memory (hypothetical) — would the GPU still need many warps?
No. Warps exist mainly to hide memory latency; if memory were instant there'd be nothing to hide, and the GPU could run with far fewer resident warps.
Recall Quick self-scoring
Which single word decides whether GPU or CPU wins a workload? ::: Whether the tasks are independent (parallel → GPU) or dependent (serial → CPU). The GPU's core advantage in one phrase? ::: More transistors spent on arithmetic and latency hidden by thread-switching instead of caches. The CPU's core advantage in one phrase? ::: Low latency for a single dependent thread via caches, branch prediction, and out-of-order execution.
See also: 6.3.01-SIMD-vs-SIMT · 6.2.02-CUDA-programming-model · 6.2.03-memory-hierarchy-GPU · 9.2.01-parallel-programming-models · 6.2.01 GPU vs CPU design philosophy (Hinglish)