Noesis
4.1.25 · D1Computer Architecture (Deep)

Foundations — GPU architecture — SIMT, warps, CUDA model

2,399 words11 min read↑ Back to topic

This page assumes you know nothing GPU-specific. We build every word and symbol the parent note leans on, one brick at a time — including the acronyms GPU, SIMD and SIMT — then hand you a checklist. Every new term is defined before it is used again.

0. The very first picture: what is a "thread"?

Before any symbols, fix the mental image.

Figure — GPU architecture — SIMT, warps, CUDA model

The picture: many identical workers, each holding the same worksheet but a different number to plug in. This single image is the seed of everything on this page — hold onto it.

1. Latency and throughput — the two clocks

The parent note says a GPU hides latency and cares about throughput. These are the two most important nouns on the whole page, so let's earn them.

2. ALU, registers, program counter — the parts of a worker

Figure — GPU architecture — SIMT, warps, CUDA model

3. SIMD — the rigid crowd (the thing SIMT is contrasted against)

The parent note repeatedly compares SIMT to SIMD. You cannot grasp the contrast without first seeing SIMD, so we define it here — before SIMT.

4. Ceiling, floor, and integer division — the index arithmetic

The parent's grid-size formula uses and "integer division." Both are symbols a 12-year-old may not have met, so we define them on a number line.

Figure — GPU architecture — SIMT, warps, CUDA model

5. Indices: blockIdx, blockDim, threadIdx

Now the four names that build the parent's master formula .

6. The hierarchy words: warp, SIMT, block, grid, SM, kernel

Only now — after threads, SIMD, and indices — do we introduce the GPU-specific grouping words.

7. Occupancy, divergence, coalescing — one line each (full treatment lives in the parent)

Prerequisite map

Thread = one worker

Warp = 32 threads lockstep

Latency vs Throughput

Latency hiding

ALU registers PC

SIMT model

SIMD rigid lanes

SM scheduler

Occupancy

Index math floor ceil

Global index formula

blockIdx blockDim threadIdx

Branch divergence

Memory coalescing

GPU SIMT warps CUDA model

Equipment checklist

Cover the right side; can you answer each before reading the parent note?

What does GPU stand for and what is it good at?
Graphics Processing Unit — a chip with many small ALUs built to run thousands of identical jobs in parallel (throughput).
What is a thread, in one image?
One worker running the same worksheet with its own scratch paper and place-marker.
Latency vs throughput?
Latency = wait time for one task; throughput = tasks finished per unit time.
What is a clock cycle?
One tick of the chip's clock — the unit hardware counts time in.
What does an ALU do?
One arithmetic/logic step; GPUs pack many of them.
What is a register?
A private ultra-fast memory slot belonging to one thread.
What is a program counter?
The worker's pointer to the line it is currently executing.
What does SIMD stand for, and what are SSE/AVX?
Single Instruction, Multiple Data; SSE (Streaming SIMD Extensions) and AVX (Advanced Vector Extensions) are CPU instruction sets that add wide-row vector ops.
What does SIMT stand for and how does it differ from SIMD?
Single Instruction, Multiple Threads; same one-instruction/many-lanes idea but each thread is programmed as a scalar and can branch independently.
What is and ?
Ceiling = 4 (round up), floor = 3 (round down).
In the grid-size formula, what are N and B?
N = number of data items to process; B = block size = threads per block.
Why does (N+B-1)/B round up in integer division?
Adding B-1 nudges any partial box up to a full box before the floor throws away the remainder.
Does blockDim count blocks?
No — it is threads PER block (a size). gridDim counts blocks.
Write the global 1D index.
.
How many threads in a warp, and why care?
32; it is the scheduling unit, so block sizes should be multiples of 32.
Warp vs block vs grid?
Warp = 32 threads lockstep; block = warps sharing on-chip memory on one SM; grid = all blocks of one launch.
What is occupancy?
Active warps ÷ max warps on an SM — measures how many spare warps hide latency.
When does branch divergence cost you?
When threads in the SAME warp take different if/else paths, forcing both to run serially.
When is a memory access coalesced?
When a warp's 32 addresses are consecutive, fitting in ~one transaction.