Foundations — ROCm - OpenCL alternatives

Before you can read the parent note on ROCm / OpenCL alternatives, you need every word and symbol it throws at you built from nothing. That is this page. We start with "what is a GPU even doing" and end with the exact formula global_id = group_id × local_size + local_id.

1. What a GPU actually is (the picture)

Look at the figure. On the left, one big square = one CPU core, working through a to-do list one step at a time. On the right, a grid of tiny squares = GPU cores, every square lit at the same instant doing "add my two numbers". This picture is the reason everything else exists: to control that grid, we need a language.

Why does the topic need this word? Because ROCm, OpenCL and CUDA are all just different spellings for "here is my kernel, please run it on a million workers".

2. Host and Device — the two computers

The figure shows two boxes joined by a thin pipe. The host box holds your data in h_A (h = host). The device box holds a copy in d_A (d = device). The pipe is the slow cable between them — this is why the parent note keeps calling cudaMemcpy / hipMemcpy / clEnqueueWriteBuffer: you must physically ship the data across that pipe before the GPU can touch it, and ship the answer back after.

3. Naming a million workers — IDs

If every worker runs the same kernel, how does worker #504 know to add the 504th pair and not the 1st? Each worker is handed a unique number.

The (0) and .x both mean "the x-direction". A grid can be 1-D, 2-D or 3-D (like a line, a sheet, or a cube of workers), so we index each direction separately.

4. Grouping the workers — Work-Groups and the NDRange

Why not just have one flat list of a million workers? Because the hardware physically runs them in fixed-size bundles: NVIDIA in warps of 32, AMD in wavefronts of 64. To match the machine, we chop the grid into equal work-groups.

The figure is the whole story of the parent note's "Work-Item Execution Model" formula. Read it left to right:

• The full amber strip = the NDRange (also called Global Work Size), total workers $G_x$.
• It is cut into cyan blocks of equal size = work-groups, each holding $L_x$ workers (the Local Work Size).
• One highlighted worker shows how its three IDs relate.

Why this last formula? It is plain chunk-numbering. If groups hold $L_x=64$ workers, then worker global number $=(\text{which group})\times 64+(\text{position inside group})$. Look at the highlighted worker in the figure: it sits in group 1, at local position 2, so its global number is $1\times 64+2=66$.

5. Where data lives — the memory hierarchy

A worker cannot reach every byte equally fast. The topic's __global / __local / __private keywords name how far away memory is.

This is why the parent note stresses that AMD has 64 KB LDS per compute unit while NVIDIA has 48–128 KB shared memory per SM: the abstraction (__local) is the same word, but the size behind it differs per vendor — and your portable code must not assume one number.

6. Why translation layers exist at all — IR and JIT

Each vendor's GPU speaks its own private machine language (NVIDIA: PTX, AMD: GCN/RDNA ISA). You do not want to rewrite your kernel for each.

The pipeline the parent shows — $\text{Source}\to \text{LLVM IR}\to \text{Device ISA}\to \text{Execution}$ — is exactly this: write once in the shared vocabulary, let IR + JIT fit it to whatever hardware you happen to own. That single sentence is the whole reason ROCm and OpenCL can escape vendor lock-in.

7. How the pieces feed the topic

Equipment checklist

Cover the right side and answer each before moving to the parent note.