6.5.6 · D5Advanced & Emerging Architectures
Question bank — Domain-specific accelerators
Before we start, a one-line refresher on the vocabulary so nothing below appears un-earned. Every symbol used later in this page is defined here first.
True or false — justify
Every answer must give the mechanism, not a verdict.
DSAs win mainly by running at a higher clock frequency than CPUs.
False. They typically clock lower; the win is massive parallelism, deleted control overhead, lower precision, and data reuse — see Systolic arrays. Currency is ops/joule, not GHz.
If a chip's peak throughput doubles, real application throughput doubles too.
False. Only if the kernel was compute-bound. If it's memory-bound (), the extra math units sit idle and nothing improves.
Using INT8 instead of FP32 always corrupts a model's accuracy.
False. For inference, millions of weights average away the rounding noise, so INT8 loses negligible accuracy. Bit-exact scientific computing is a different domain that does not tolerate it.
A systolic array reduces the number of multiply-adds needed for a matmul.
False. It still does MACs; what it reduces is memory traffic per op by reusing each loaded value times inside the grid.
DSAs make CPUs obsolete for good systems.
False. DSAs are narrow — anything outside their domain runs slowly or not at all. Real machines are heterogeneous: the CPU orchestrates and the DSA runs the hot kernel.
The end of Dennard scaling is the same event as the end of Moore's Law.
False. Moore's Law (transistor count) limped on; Dennard scaling (constant power density as transistors shrink) died ~2006. We got more transistors we couldn't all power — that gap is Dark silicon.
A software-managed scratchpad is just a cache with a different name.
False. A cache spends transistors/energy on tags, replacement logic, and coherence. A scratchpad drops all of that for deterministic latency — the software decides what lives there.
Lower precision helps energy but not throughput.
False. Both. A multiplier's cost scales roughly with in bit-width , so halving bits gives ~4× more units in the same area and ~4× lower energy per op.
Raising memory bandwidth always raises attainable performance.
False. Only while memory-bound. Once you're compute-bound; more bandwidth is then wasted, just as more MACs are wasted when memory-bound.
Spot the error
Each statement contains one flaw. Name it. (Recall = operations, = bytes moved, .)
"Our kernel does FLOPs and moves bytes, so intensity is ."
Inverted. Intensity is operations per byte, . Flipping it would make data-reuse-heavy kernels look memory-bound, the opposite of the truth.
"Peak throughput is 90 TOPS and the bandwidth ceiling is 15 TOPS, so attainable is 90 because we take the max."
Wrong operator. Roofline takes the min: TOPS. You're limited by whichever wall you hit first, not the friendlier one.
"A CPU's integer add wastes energy because the ALU adder is expensive."
Backwards. The adder itself is a tiny fraction of a picojoule; the waste is fetch + decode + register-file + control, which can cost 10–100× the op. That overhead is what specialization deletes.
"Since a systolic array is compute-bound, its arithmetic intensity is ."
Contradiction. It's compute-bound because intensity is (each value reused across the array dimension). intensity is the memory-bound naive case.
"Doubling a single core's transistors doubles its speed — that's why we build bigger cores."
Violates Pollack's Rule: single-core performance grows only as , so double the transistors gives ~1.4×, not 2×. That diminishing return is a reason to specialize instead.
"Training must stay entirely in FP32 or gradients explode, so DSAs can't help training."
Overstated. Mixed precision training keeps only the sensitive parts (e.g. accumulations, master weights) high-precision while doing the bulk math in bf16/FP16 — DSAs accelerate training routinely.
Why questions
State the underlying reason, not just "because."
Why does arithmetic intensity , not raw work , decide whether a DSA helps?
Because time is ; if the term dominates, adding compute (raising ) changes nothing — only raising reuse () shortens the memory term.
Why does a systolic array "load once, then stream"?
To amortize an expensive memory load over cheap on-chip reuses, pushing intensity from to and turning a memory-bound problem into a compute-bound one.
Why is ops/joule the binding metric in the dark-silicon era, not clock speed?
Because we can fit more transistors than we can power/cool at once, so the limit is energy budget; whoever does more useful work per joule can light up more of the chip.
Why can DSAs exploit low precision that a general CPU can't easily take?
A CPU must stay correct-to-the-bit for arbitrary workloads; a DSA exploits a domain tolerance (statistical averaging in neural nets) that guarantees the noise is harmless there.
Why do DSAs drop out-of-order execution and branch prediction?
Their kernels are regular and predictable (dense matmul has no data-dependent branches), so speculation buys nothing but costs area and energy — deleting it is pure profit.
Why does building "many specialized blocks" beat "one huge fast core"?
By Pollack's Rule a huge core gives sub-linear return per transistor, and by Dark silicon you can't power everything anyway — so light up only the specialized block the current task needs.
Why is a GPU still more general than a TPU?
A GPU keeps programmable SIMD lanes and caches for many parallel workloads; a TPU hard-wires the systolic matmul dataflow, trading that flexibility for even higher matmul efficiency.
Edge cases
The scenarios that break naive intuition.
What happens to a DSA's advantage when the kernel is tiny (fits in registers, few ops)?
The advantage collapses — Amdahl's Law dominance shifts to launch/overhead, and there's no data-reuse to amortize; a CPU may finish sooner than it takes to hand work to the accelerator.
What if arithmetic intensity is extremely high (huge reuse)?
You're firmly compute-bound and pinned at ; now the only way up is adding MACs or raising precision throughput — bandwidth upgrades are wasted.
What if exactly equals ?
You sit precisely at the roofline's "ridge point" — perfectly balanced, using 100% of both walls. Any further optimization must move both compute and memory together.
What happens if you feed a systolic array data that doesn't fill the grid (a matmul on a array)?
Most MAC cells idle; utilization plummets and effective TOPS is far below peak — small or oddly-shaped problems waste the array, a real reason DSAs favor large batches.
What is the accuracy story for INT8 at the boundary — outlier weights with large magnitude?
Uniform Quantization can clip or coarsely bin outliers, hurting accuracy; the fix is per-channel scales or keeping outlier layers in higher precision, i.e. mixed precision even at inference.
What happens to DSA benefit as the non-accelerated ("serial") fraction of a workload grows?
Amdahl's Law caps the total speedup: if the DSA can't touch fraction , overall speedup is bounded by no matter how fast the accelerated part becomes.
At zero data reuse (every operand fetched exactly once), what is the best a systolic array can do?
It degenerates to the memory-bound case — the grid gives no help because the whole point (reuse) is absent; you're limited purely by bandwidth .
Recall Self-check: the one sentence that ties it all together
If you can explain this line, you've got the topic ::: DSAs win by deleting general-purpose overhead and reusing data on-chip to raise arithmetic intensity, so the Roofline model lets many low-precision MACs stay fed and deliver far more ops/joule — but only within their narrow domain.