Foundations — Heterogeneous computing concepts

Before you can read the parent note, you need to own every word and symbol it throws at you. This page builds them one at a time, from nothing. Read top to bottom — each block only uses ideas already defined above it.

1. A "processor" and what it does

If you put many desks in one room, you have many cores. If all the desks are identical, that room is homogeneous. If some desks are big-and-clever and others are small-and-many, the room is heterogeneous — that word is literally Greek for "different kinds".

Figure s01 below draws exactly this contrast: on the left, one large blue desk = the CPU (a single powerful worker); on the right, a grid of small green desks = the GPU (many simple workers). As you read the next callout, look at the two blocks and notice that the blue block is one box while the green block is twenty-four — that size-vs-count trade-off is the whole picture.

2. FLOP, FLOPS, and why the letter S matters

The parent note counts work in "FLOPs". Two nearly-identical words hide here — mixing them up ruins every formula.

Prefixes you will see (each is ×1000 of the last):

So "10 TFLOPS" = ten trillion operations every second.

3. Time = Work ÷ Speed (the master equation)

4. Bandwidth $B$, data size $D$, and the "$2D/B$" tax

A GPU cannot touch data that sits in the CPU's memory. The data must be copied over a wire first. That wire is slow, and the copy costs time.

Same master equation, new names: time to move the box = box size ÷ pipe width = $D/B$.

Figure s02 below makes the round trip concrete. The blue box (CPU memory) and green box (GPU memory) sit at the two ends of a grey pipe of width $B$. The yellow arrow is the send leg (CPU→GPU) costing $D/B$; the red arrow underneath is the return leg (GPU→CPU) costing another $D/B$. Follow both arrows and you literally count the two trips that add up to the tax written at the bottom.

5. Arithmetic intensity — the make-or-break ratio

Now the punchline. Two chips race, but only the fast chip pays the transfer tax. When is paying it worth it?

Figure s03 below plots that ratio against problem size $n$. The green curve (matrix × matrix) climbs forever because its intensity grows like $n$; the red curve (vector add) stays flat and low. The yellow dashed line is the break-even threshold from the transfer tax — you only "win" in the shaded green region above the line, which the green curve enters and the red curve never reaches. That single picture explains why some workloads belong on the GPU and others never do.

6. Fractions $f_s$ and $f_p$ — splitting a program in two

Amdahl's Law needs one more idea: a program is part "must-be-sequential" and part "can-be-parallel".

Now we define the last symbol the ceiling needs — the normalized transfer overhead.

Figure s04 below shows the payoff. Each curve is the overall speedup as the GPU gets faster (moving right). Notice every curve flattens into a horizontal ceiling — the dotted line of the same colour — no matter how far right you go. A small serial fraction (green, $f_s=0.05$) has a high ceiling; a large one (red, $f_s=0.5$) is capped low. That flattening is Amdahl's Law.

7. Performance-per-watt — why accelerators exist at all

Prerequisite map

Each foundation on the left feeds a formula in the middle, and all of them feed the parent topic on the right: Heterogeneous Computing. When you also want to express these splits in code, read Parallel Programming Models.

Equipment checklist

Cover the right-hand side and see if you can answer each before revealing it.