Intuition The one idea behind this whole topicA CPU and a GPU both do arithmetic, but they answer different questions : a CPU asks "how fast can I finish one job?" while a GPU asks "how many jobs can I finish per second ?". Every symbol below — latency, throughput, cores, transistor budgets, warps — exists only to make that one trade-off precise and measurable.
Before you can read GPU vs CPU design philosophy you must be able to read every mark on the page . This note takes each symbol, term, and picture the parent uses and builds it from nothing. Read top to bottom; each block only uses words defined above it.
Everything starts with one word: a task . A task is one self-contained piece of work — colour one pixel, compare two numbers, add two floats.
There are two completely different ways to measure how good a machine is at tasks. Confusing them is the single biggest mistake in this topic, so we build both pictures first.
Latency is the time from the moment one task starts to the moment that same task finishes . Picture a single stopwatch that you start when the runner leaves the blocks and stop when they cross the line. Small latency = the one job finished quickly.
Throughput is the number of tasks finished per unit of time — a rate , not a duration. Picture standing at the finish line counting how many runners cross per minute, ignoring how long any single runner took. Big throughput = many jobs done per second.
Intuition Why we need BOTHA conveyor belt in a factory can have high latency (each item takes 10 minutes to travel the belt) yet high throughput (one finished item drops off every second, because 600 items are on the belt at once). Low latency does not imply high throughput, and vice-versa. The parent note's entire argument is: CPUs chase low latency, GPUs chase high throughput.
Why does the topic need two separate words? Because a design choice that helps one can hurt the other. If you didn't have two names, "the GPU is slower but faster" would sound like nonsense — with the two names it becomes "higher latency, higher throughput", which is exactly true.
The parent note writes formulas full of letters. Each letter is just a count or a rate . Here is the whole alphabet you need.
Definition The counting symbols
N a count — a plain whole number. N instructions N tasks N cores P number of parallel processors (workers acting at the same time). P N cores P parallelism .t time for ONE thing . t inst t task T total time for the whole job. Units: seconds.f clock clock frequency = how many steps the chip takes per second, measured in GHz (billions per second).
Intuition Why a fraction likeT = P N × t task N = 12 t task = 30 P = 3 N / P = 4 3 12 × 30 = 120 divides the work among the workers ; the × t task
Why does the topic need division here and not, say, subtraction? Because adding more workers splits the pile — a multiplicative/ratio effect — not a fixed discount. Division is the only operation that captures "share the pile evenly".
The CPU model reads T C P U = I P C × f clock N instructions
Definition Cycle, clock frequency, and IPC
A cycle is one tick of the chip's internal clock — the smallest heartbeat of the processor.
f clock IPC = Instructions Per Cycle = how many instructions the chip finishes during one tick . If IPC = 4, the chip retires 4 instructions every heartbeat.
Why does the topic care about IPC at all? Because the parent's central claim is that a CPU spends most of its transistors just to push IPC higher on one thread. If you didn't know what IPC meant, "control logic exists to maximize IPC" would be an empty sentence.
The parent gives "transistor budgets" for CPUs and GPUs. You need three ideas: transistor, die, and budget.
Definition Transistor, die, budget
A transistor is the tiniest switch on a chip — the atom of digital hardware. Billions of them wired together make everything.
The die is the single flat rectangle of silicon the chip is carved from. It has fixed area .
A transistor budget B fixed — spending transistors on one feature means fewer for another.
Intuition The budget is a pie you must sliceThink of B compute (units that do arithmetic), cache (fast local memory), and control (logic that predicts branches, reorders instructions). A CPU cuts big cache + control wedges and a small compute wedge. A GPU cuts a huge compute wedge and tiny cache/control wedges. Same pie, opposite slicing — that is the design philosophy in one picture.
A cache is a small pool of very fast memory sitting right next to the compute units, holding copies of recently-used data so the chip doesn't have to make the long trip to main memory (DRAM). CPUs stack three levels — L1 (smallest, fastest), L2, L3 (largest, shared).
The parent's arithmetic 36.5 × 1 0 6 × 8 × 6 megabytes → bits → transistors . One byte = 8 bits, and an SRAM cache cell needs about 6 transistors per bit. So "how many transistors does 36.5 MB of cache cost?" = megabytes × 8 × 6. Nothing mysterious — it's unit conversion.
The parent splits all workloads into two kinds. This distinction decides whether the GPU's "share the pile" trick even works.
Definition Data dependencyTwo tasks have a data dependency when one needs the answer of the other before it can start . Written I n I n − 1 n n − 1 cannot do them at the same time — like you can't ice a cake before it's baked.
Definition Embarrassingly parallelA workload is embarrassingly parallel when the tasks have no dependencies at all — each can run in total isolation. Colouring pixel ( x 1 , y 1 ) ( x 2 , y 2 ) ideal case for the "pile ÷ workers" formula, because you can use as many workers as you like.
Common mistake Parallel workers can't rescue sequential workBinary search (Example 2 in the parent) does 30 steps where each step needs the previous step's comparison — a chain of dependencies. Throwing 4000 GPU cores at it helps zero , because 3999 of them would just wait. That is why the parent's binary-search example lets the CPU win . Independence, not core count, is what the GPU needs.
The last cluster of symbols is how a GPU deals with slow memory without big caches.
Definition Thread, warp, SM, occupancy
A thread is one running task — the software worker executing one instruction stream.
A warp is a fixed bundle of 32 threads that a GPU always steers together: all 32 run the same instruction at the same tick. This scheme is called SIMT (Single Instruction, Multiple Threads).
An SM (Streaming Multiprocessor) is one cluster of GPU cores that holds many warps at once.
Occupancy = the fraction (0 to 1) of the GPU's cores that are actually busy computing right now, not stalled waiting.
Intuition Latency hiding = keep spare runners warmed upWhen one warp asks DRAM for data, it must wait hundreds of cycles. Instead of sitting idle, the SM instantly switches to a different warp that is ready to run. With dozens of warps loaded, there is always someone ready, so the long memory wait is hidden behind other people's work . This is why a GPU can skip the giant cache a CPU needs: it trades storage (cache) for spare workers (warps).
Definition Branch divergenceInside one warp all 32 threads must run the same instruction. If an if sends some threads one way and others another, the GPU must run both paths one after the other , switching off the threads that don't belong — this serial slowdown is branch divergence . A CPU avoids this pain with a branch predictor ; a GPU has none, so divergence hurts.
data dependency vs independence
GPU vs CPU design philosophy
Where these threads continue in the vault: the memory story deepens in 6.2.03-memory-hierarchy-GPU , the warp/SIMT idea in 6.3.01-SIMD-vs-SIMT , the programming side in 6.2.02-CUDA-programming-model and 9.2.01-parallel-programming-models , the limits of speedup in 9.1.02-Amdahls-law , and the ancestor of all this hardware in 6.1.01-von-Neumann-architecture .
Cover the right side; say your answer aloud before revealing.
What is latency, in one sentence? The time from when a single task starts until that same task finishes.
What is throughput, and how does it differ from latency? The number of tasks finished per unit time — a rate, whereas latency is a single duration.
Can a machine have high latency AND high throughput at once? Yes — a conveyor belt: each item takes long (high latency) but one drops off every second (high throughput).
In T = P N × t t a s k P Sharing the pile of N P
What does IPC stand for and mean? Instructions Per Cycle — how many instructions the chip finishes in one clock tick.
Why is I P C × f c l oc k (instructions per tick) × (ticks per second) = instructions per second.
What is a transistor budget B The fixed number of transistors on the die; spending them on cache/control leaves fewer for compute.
Convert cache size to transistors: what is the recipe? megabytes × 8 (bits per byte) × 6 (transistors per SRAM bit).
What is a data dependency? When one task needs another task's result before it can start, forcing sequential order.
What does "embarrassingly parallel" mean? Tasks with no dependencies at all, each runnable in isolation.
How big is a warp, and what does SIMT mean? 32 threads; Single Instruction Multiple Threads — all 32 run the same instruction together.
How does a GPU hide memory latency without big caches? It switches to another ready warp while one warp waits, keeping cores busy.
What is branch divergence? When threads in a warp take different if paths, forcing the GPU to run each path serially.
