Before you can read the parent topic, you need a small vocabulary of physical objects inside the chip. We build each one from nothing — a picture first, the word second, the symbol last.
A thread on its own is unremarkable — slower than a CPU thread, in fact. Its power only appears when you have many. That is why every symbol below counts threads, or groups of threads, rather than caring about any one of them.
Here is the first idea the parent note assumes you already own. Threads are not scheduled one at a time. They are bundled into fixed groups of 32, and all 32 execute the same instruction at the same moment, each on its own data.
Whenever the parent writes "active warps", it means: groups of 32 threads currently living on the hardware, ready to be told what to do.
Every quantity on this page is measured per SM. "64 warps per SM" means one kitchen can hold at most 64 warps' worth of workers standing at its counters at once.
Latency and throughput are two different questions about the same pipeline, and confusing them is the classic mistake. A pipeline can have long latency yet high throughput — like an assembly line where each car takes hours (latency) but one rolls off every minute (throughput).
Now we can build the parent note's central formula. The question is: to keep something busy every cycle, given that each request takes L cycles to return, how many requests must be in flight at once?
Reading the picture (s03): each row is one warp's timeline. A warp fires a load (red) then waits L cycles. If you stack enough warps, at every column (cycle) at least one warp is doing useful issuing — no empty column, no idle SM.
Worked plug-in. One scheduler issues λ=1 instruction per cycle. A DRAM load costs L=400 cycles. Then:
N=1×400=400 instructions in flight.
That is the naïve single-scheduler number the parent quotes. It overshoots in practice because concurrency is supplied SM-wide (4 schedulers, plus several instructions in flight per warp), so 16–32 active warps usually suffice.
Now every symbol is earned, so we can state the topic's namesake quantity.
Occupancy matters only because of Little's Law: more active warps means more requests in flight means more of the fixed latency L gets hidden — up to the point where you already have enough, after which extra warps do nothing.
Test yourself — reveal only after answering aloud.
What is a warp, in one sentence?
A group of exactly 32 threads that execute the same instruction in lockstep, each on its own data.
What does "per SM" mean and why do we always measure there?
Per Streaming Multiprocessor — one self-contained kitchen with its own registers, shared memory, and schedulers; occupancy and warp limits are all defined per SM.
Why is switching between warps essentially free?
Each warp's registers stay parked on-chip, so nothing is copied — the scheduler just points at a different ready warp next cycle.
What is latency L, in what units?
The number of cycles you must wait between requesting something and receiving it; measured in clock cycles.
Can adding more warps shrink the latency of a single load?
No — L is fixed; extra warps only let you overlap many waits so the SM stays busy.
What question does throughput answer that latency does not?
How many operations complete per cycle (a rate), versus how long one operation takes (a duration).
State Little's Law and name each symbol.
N=λL: N = operations in flight, λ = throughput (ops/cycle), L = latency (cycles).
Why does ⌊⋅⌋ appear in the occupancy formulas?
Warps and blocks are indivisible whole units; leftover space too small for one more whole unit is wasted, so we round down.
Define occupancy as a formula.
Active warps per SM divided by the maximum warps per SM, expressed as a percentage.
