Foundations — Domain-specific accelerators

This page assumes you know nothing but basic arithmetic and what a computer chip roughly is. Every symbol the parent note throws at you is built here from the ground up. Read top to bottom — each block earns the next.

0. The picture of a chip (so the rest makes sense)

A chip is a flat square of silicon. On it are billions of tiny switches called transistors. Groups of transistors form blocks — some do arithmetic, some fetch instructions, some store data. Everything below is about which blocks are worth building and why.

Figure s01 — the chip floor-plan. Look at the six labelled boxes: the amber box is the ALU where the actual math happens; every white box (fetch, decode, branch prediction, reorder, cache) is overhead — work spent guessing and shuffling, not computing. Notice how few boxes are amber. That imbalance is the whole reason DSAs exist: delete the white boxes for one fixed job and almost all the energy goes to real math.

1. Energy and power — the units everything is measured in

Before we can say "DSAs win", we need to say win at what. The answer is energy per operation, so we must nail down two words that people confuse.

2. Power density — the wall that started it all

Now the historical rule that used to save us:

Figure s02 — why shrinking stopped being free. Follow the cyan line first: during the Dennard era, generation after generation, power density stays flat — free lunch. At the dashed line (~2006) the amber line takes over and climbs steeply: each new generation now pours more heat into the same square millimetre. The amber arrow points to the consequence — you hit the power-density ceiling and must leave transistors dark.

3. Diminishing returns from a single core — two named rules

Even if we could power a big core, making one core bigger pays off badly.

4. The building blocks of the accelerator itself

Now the vocabulary of what a DSA is made of.

Figure s03 — load once, stream, reuse. The white arrows drop weights down into the grid — each weight is loaded once and stays. The amber arrows push activations rightward, one column of cells to the next. Follow a single weight sitting in a cell: every activation that flows past it triggers one MAC, so that one loaded number is reused $N$ times across the row. That reuse — many operations per byte fetched — is exactly what the next section calls arithmetic intensity.

5. Precision — spending fewer bits per number

6. Arithmetic intensity and the Roofline — the decision tool

This is the most symbol-heavy idea in the parent note, so we build every letter.

Figure s04 — reading the roofline. Two ceilings form the "roof": the sloped amber line (slope $=B$, the memory limit — how fast bytes arrive) and the flat white line ($P_{\text{peak}}$, the compute limit). Your kernel is a vertical position on the intensity axis. The amber dot at $I=50$ lands on the sloped part, well below $P_{\text{peak}}$: you are memory-bound, capped at 15 TOPS. The dotted "ridge" at $I=300$ is where the two ceilings meet — only kernels to the right of it are compute-bound. Moving the dot rightward (more reuse) is the only way to climb here.

Prerequisite map

Equipment checklist

Cover the right side and answer aloud. If any one stumps you, re-read its section above.