GPU Architecture

Question 1 (4 marks)

Any two valid contrasts, 2 marks each (1 for difference, 1 for reasoning):

• CPU: few, powerful cores optimized for low-latency serial execution; large caches and complex control logic (branch prediction, out-of-order). Why: to run single threads as fast as possible.
• GPU: thousands of simpler cores optimized for high throughput / parallelism; small caches, more ALUs, latency hidden by massive threading. Why: to maximize total work across data-parallel tasks.
• Also acceptable: CPU spends transistors on control/cache; GPU spends them on arithmetic units. CPU = latency-oriented; GPU = throughput-oriented.

Marks: 2 valid points × 2 = 4.

Question 2 (5 marks)

• (a) SM: a processing unit on the GPU containing CUDA cores, schedulers, registers, and shared memory; executes thread blocks. (1)
• (b) CUDA core: a scalar arithmetic/logic execution lane that performs one thread's operation. (1)
• (c) Warp: a group of 32 threads executed in lockstep (SIMT) by an SM. (1)
• (d) Thread block: a group of threads that execute on the same SM and can cooperate via shared memory and synchronization. (1)
• (e) Grid: the entire collection of thread blocks launched by a kernel. (1)

Question 3 (5 marks)

• SIMT definition (3): A single instruction is issued to a warp of threads; each thread has its own registers and program counter/state and operates on its own data. Threads execute the same instruction stream in lockstep but on different data elements.
• Distinction from SIMD (2): SIMD operates on fixed-width vectors with one control flow; the programmer manages vector lanes explicitly. SIMT exposes individual threads with independent addressing and can handle divergent branches per thread (at a performance cost), giving a scalar programming model over vector hardware.

Question 4 (6 marks)

• (a) Total threads $= 256 \times 512 = 131072$. (2)
• (b) Warps per block $= 512 / 32 = 16$. (2)
• (c) Total warps $= 256 \times 16 = 4096$ (or $131072/32$). (2)

Question 5 (5 marks)

Order (3 marks — 1 per correct level in order): Registers → Shared memory (L1) → Global memory. Characteristics (2 marks, any valid):

• Registers: per-thread, fastest, smallest.
• Shared memory: per-block scope, on-chip, low latency.
• Global memory: device-wide scope, largest, highest latency (off-chip DRAM).

Question 6 (6 marks)

• Definition (3): Coalesced access occurs when threads in a warp access contiguous, aligned memory locations so that the hardware combines them into a minimal number of memory transactions, maximizing bandwidth utilization.
• Calculation (3): $32 \text{ threads} \times 4 \text{ bytes} = 128$ bytes, served by a single 128-byte transaction. Efficient because every byte fetched is used (no wasted bandwidth); one transaction instead of up to 32 separate ones.

Question 7 (5 marks)

• Definition (3): Shared memory is divided into equally-sized banks (32). Successive 32-bit words map to successive banks. A bank conflict happens when multiple threads in a warp access different words in the same bank simultaneously; these accesses cannot be serviced in parallel.
• Conflict result (2): With 8 threads hitting the same bank (different words), the accesses are serialized into 8 transactions → an 8-way bank conflict, reducing effective throughput by a factor of 8. (Broadcast of the same word is an exception and causes no conflict.)

Question 8 (6 marks)

• (a) Resident threads $= 6 \times 256 = 1536$. (2)
• (b) Occupancy $= 1536 / 2048 = 0.75 = 75\%$. (2)
• (c) Relation (2): Higher occupancy provides more resident warps for the scheduler to switch to while others wait on memory/compute latency, improving latency hiding and keeping execution units busy. (Note: max occupancy is not always necessary for peak performance.)

Question 9 (4 marks)

• Definition (2): Warp divergence occurs when threads in the same warp follow different control-flow paths (branches); since a warp executes one instruction at a time, both paths cannot run simultaneously.
• Penalty (2): The hardware executes each branch path serially, masking off (disabling) threads not on the current path. With a 16/16 split, the if and else paths run one after another — roughly doubling execution time for that branch region (effective utilization ~50% during divergence).

Question 10 (4 marks)

• (a) Tensor Core (2): a specialized execution unit that performs matrix multiply–accumulate (MMA), e.g. $D = A \times B + C$ on small matrix tiles, accelerating dense linear algebra / deep-learning workloads.
• (b) Open framework (2): ROCm (AMD ecosystem) or OpenCL (cross-vendor / Khronos), or SYCL/oneAPI (Intel). Any one with correct association.

[
  {"claim":"Total threads = 256*512 = 131072","code":"result = (256*512 == 131072)"},
  {"claim":"Warps per block = 512/32 = 16","code":"result = (512/32 == 16)"},
  {"claim":"Total warps = 256*16 = 4096 = 131072/32","code":"result = (256*16 == 4096 and 131072/32 == 4096)"},
  {"claim":"Coalesced transfer 32*4 = 128 bytes","code":"result = (32*4 == 128)"},
  {"claim":"Occupancy 6*256/2048 = 0.75","code":"result = (sympy.Rational(6*256,2048) == sympy.Rational(3,4))"}
]