Wafer-scale engines (Cerebras-style): Worked examples

This page is the "put numbers in and turn the crank" companion to Wafer-scale engines (Cerebras-style). Before touching a single number, we lay out every kind of question the topic can ask, so that when you meet one in an exam you already know which cell it lives in — and no scenario surprises you.

Every symbol used here was first defined in the parent note. We re-state each one the instant it appears, so you never have to scroll back.

Symbols, defined before we use them

Because the scenario matrix below already names formulas, we pin every symbol down first, so no letter appears before its plain-words meaning.

Definition The quantities we compute with (read this before the matrix)

$D$ — defect density, defects per mm² of silicon. Picture tiny random dots sprinkled on the wafer.
$A$ — die area, mm². The rectangle we cut out of the wafer.
$Y$ — yield, the probability that one die is defect-free (a number between $0$ and $1$ ).
$λ$ (Greek letter "lambda") — the mean (average) number of killing defects expected to land inside one die. It equals $D \cdot A$ . This is the single knob the yield formula turns on.
$M$ — number of cores physically built on the wafer.
$N$ — number of good cores we actually need for the job.
$f$ — fraction of cores allowed to fail (a number between $0$ and $1$ ).
$B_{cor e}$ — bandwidth one core can push to its neighbours, in GB/s.
$B_{t o t}$ — total aggregate bandwidth, the sum of $B_{cor e}$ over all cores.
$S_{cor e}$ — SRAM (fast on-chip memory) held by one core; $S_{t o t}$ — the sum over all cores.
$z$ — fraction of activations that are zero (the sparsity level), between $0$ and $1$ .

The scenario matrix

Think of the whole topic as a grid. Each row is a quantity you might be asked to compute; each column is a regime (tiny/normal/extreme, or a degenerate edge). We must cover every filled cell below. Every symbol in the table was just defined directly above.

Cell	Quantity	Regime being tested	Worked in
A	Monolithic yield $Y = e^{- D A}$	Small die (near 1)	Ex. 1
B	Monolithic yield $Y = e^{- D A}$	Whole-wafer die (near 0, the "killer")	Ex. 1
C	Yield limit	Degenerate: $D \to 0$ and $A \to 0$	Ex. 2
D	Over-provisioning $M \geq N / (1 - f)$	Normal fail fraction	Ex. 3
E	Over-provisioning	Edge: $f \to 0$ and $f \to 1$	Ex. 3
F	Aggregate bandwidth $B_{t o t} = M \cdot B_{cor e}$	Order-of-magnitude compare vs GPU	Ex. 4
G	Mesh hop latency	Neighbour (best) vs corner-to-corner (worst)	Ex. 5
H	Aggregate SRAM capacity	"Does the model fit?" word problem	Ex. 6
I	Sparsity speedup	Fraction of zeros skipped	Ex. 7
J	Combined exam twist	Yield and capacity and bandwidth together	Ex. 8

Before any arithmetic, here is the whole matrix drawn as pictures. Each panel below is the shape of one formula — glance at these first, then the numbers will just be points on curves you have already seen.

Figure — Wafer-scale engines (Cerebras-style)

Alt text: four small pastel panels. Top-left — the yield curve $Y = e^{- D A}$ falling from 1 toward 0 as die area grows, with the small-die point (green, high) and whole-wafer point (coral, floor-crushed) marked. Top-right — the provisioning factor $1/ (1 - f)$ rising gently then shooting to infinity as $f \to 1$ . Bottom-left — aggregate bandwidth as a tall stack of 850k parallel links towering over a single short GPU bar. Bottom-right — a mesh grid with a short green best-case hop and a long coral worst-case corner-to-corner path.

Example 1 — Cells A & B: the yield killer, small die vs whole wafer

The figure below is the shape of the answer: the yield curve $Y = e^{- D A}$ and where our two dies land on it.

Alt text: yield curve $Y = e^{- D A}$ plotted against die area $A$ . A green dot marks the small 800 mm² die still low on the curve; a coral dot marks the 46,000 mm² wafer crushed flat on the zero axis, with a dashed guide showing how fast the curve collapses.

Step 1 — Compute the mean defect count $λ = D A$ for each. Why this step? Recall from the symbols box that $λ$ is the average number of killing defects expected in one die, and it equals $D \cdot A$ . The exponential $e^{- D A}$ only makes sense once we know this average; it is the single knob everything depends on.

$λ_{1} = 0.02 \times 800 = 16, λ_{2} = 0.02 \times 46, 000 = 920$

Step 2 — Apply the Poisson zero-defect probability. Why this step? A die is good only when it catches zero killing defects, and the Poisson probability of zero events with mean $λ$ is $e^{- λ}$ (from the parent note's derivation).

$Y_{1} = e^{- 16} \approx 1.1 \times 1 0^{- 7}, Y_{2} = e^{- 920} \approx 1 0^{- 400}$

Step 3 — Interpret against the matrix cells. Why this step? Numbers alone don't teach; the regime does. Cell A (small die) is already terrible; cell B (whole wafer) is astronomically, unimaginably zero — this is the coral dot pinned to the floor in the figure.

Recall What does

$1 0^{- 400}$ mean physically? More wafers than atoms in the universe would have to be made before one came out perfect ::: which is exactly why a defect-free monolithic wafer is impossible, forcing the redundancy trick.

Verify: Units check — $D A$ is $(1/mm^{2}) (mm^{2}) =$ dimensionless, correct for an exponent. Sanity: bigger $A \Rightarrow$ smaller $Y$ , and both are between $0$ and $1$ . ✓

See Yield & Defect Density Models for the full derivation lineage.

Example 2 — Cell C: the two degenerate limits of yield

Look again at the yield curve in the Example 1 figure: both degenerate corners walk the curve back to its left edge, where $A$ or $D$ vanishes and the height returns to $1$ .

Step 1 — Substitute the limit into the exponent. Why this step? Degenerate inputs test whether the formula stays sane. In both cases the exponent $D A \to 0$ .

$Y = e^{- 0} = 1$

Step 2 — Read the meaning. Why this step? A yield of exactly $1$ means "always works." A flawless process (no defects) or an infinitesimal die (nothing to hit) both guarantee success — the formula correctly returns certainty.

Verify: $e^{- D A}$ is continuous and $e^{0} = 1$ ; both limits agree. ✓

Example 3 — Cells D & E: over-provisioning cores, normal and edge

The figure shows the provisioning factor $1/ (1 - f)$ : gentle near $f = 0$ , then a wall as $f \to 1$ . Our three cases are three dots on this curve.

Alt text: curve of the over-provision factor $1/ (1 - f)$ versus fail fraction $f$ . Flat and near 1 on the left; a green dot at $f = 0.10$ just above 1.1; a lavender dot at $f = 0$ exactly at 1; and a coral arrow at $f \to 1$ where the curve shoots to infinity.

Step 1 — Plug $f = 0.10$ into the over-provision formula. Why this step? The factor $\frac{1}{1 - f}$ is the "insurance premium" — how many spare cores to build so that after $f$ die, enough survive. On the figure this is the green dot, barely above 1.

$M \geq \frac{850 , 000}{1 - 0.10} = \frac{850 , 000}{0.9} \approx 944, 444$

So build ~944k cores to guarantee 850k survivors — about 11 % extra.

Step 2 — Take the edge $f \to 0$ . Why this step? Zero failures should demand no spares — the lavender dot sits exactly at height 1.

$M \geq \frac{N}{1 - 0} = N = 850, 000$

Step 3 — Take the edge $f \to 1$ . Why this step? If almost every core fails, you need almost infinitely many. The denominator $1 - f \to 0$ , so $M \to \infty$ — the coral wall in the figure, the formula screaming "this process is unusable."

$lim_{f \to 1} \frac{N}{1 - f} = + \infty$

Verify: monotonic — as $f$ rises $0 \to 1$ , $M$ rises $N \to \infty$ . Check (a): $944, 444 \times 0.9 = 849, 999.6 \geq 850, 000$ ? It rounds up to satisfy the inequality; using exact $944, 444. \overline{4}$ gives exactly $850, 000$ . ✓

Example 4 — Cell F: aggregate bandwidth vs a GPU

The figure makes the scale visceral: a towering stack of parallel links beside a single short GPU bar.

Alt text: bar chart on a log scale comparing WSE aggregate bandwidth (850,000 GB/s, tall lavender bar) against a single GPU link (900 GB/s, short coral bar), with the ~944× ratio annotated by an arrow.

Step 1 — Sum the links (they run in parallel). Why this step? In a mesh every link carries data simultaneously, so total = simple sum. This additivity is the reason wafer-scale bandwidth is huge — Network-on-Chip (NoC) & Mesh Topologies.

$B_{t o t} = 850, 000 \times 1 GB/s = 8.5 \times 1 0^{5} GB/s = 850 TB/s$

Step 2 — Divide by the GPU number. Why this step? An absolute number means nothing; the ratio is the headline "why it wins."

$\frac{B _{t o t}}{B _{GP U}} = \frac{850 , 000}{900} \approx 944$

So ~944×, i.e. roughly the "1000×" quoted in the parent note.

Verify: unit check — GB/s summed gives GB/s; ratio is dimensionless. $850 TB/s = 850, 000 GB/s$ (using $1 TB = 1 0^{3} GB$ , decimal SI). ✓ Relates to the Dennard Scaling & the Memory Wall problem this solves.

Example 5 — Cell G: mesh hop latency, best vs worst case (geometric)

Alt text: a mesh grid of lavender core squares. A short green arrow marks the best case — a single hop between neighbours (1 ns). A long coral staircase path traces the worst case, corner to corner along rows then columns (~2000 hops, ~2 µs). The start core and far corner are highlighted in butter yellow.

Step 1 — Convert clock speed to a per-hop time. Why this step? Latency lives in seconds, not hertz. One cycle at $1$ GHz is the reciprocal of the frequency.

$t_{h o p} = \frac{1}{1 \times 1 0 ^{9} Hz} = 1 \times 1 0^{- 9} s = 1 ns$

Step 2 — Best case: one hop. Why this step? Neighbours are adjacent (the green arrow in the figure) — a single link. This is the common case for local, streaming traffic.

$t_{b es t} = 1 \times t_{h o p} = 1 ns$

Step 3 — Worst case: Manhattan distance across the grid. Why this step? On a 2-D mesh you can only move along rows and columns, so corner-to-corner (the coral staircase in the figure) of an $L \times L$ grid is $(L - 1) + (L - 1) = 2 L - 2 \approx 2000$ hops for $L = 1000$ — not the straight-line diagonal.

$t_{w or s t} \approx 2000 \times 1 ns = 2000 ns = 2 μ s$

Step 4 — Compare. $\frac{t _{w or s t}}{t _{b es t}} = 2000$ Even the worst on-wafer latency (~2 µs) rivals the best inter-server network hop — and locality keeps nearly all real traffic at 1 ns.

Verify: $1/1 0^{9} = 1 0^{- 9}$ ✓; $2000 \times 1 0^{- 9} = 2 \times 1 0^{- 6}$ s $= 2 μ$ s ✓. Manhattan distance for $L = 1000$ $= 1998$ , rounded to 2000 for the estimate. See Systolic Arrays & TPUs for why nearest-neighbour dataflow is efficient.

Example 6 — Cell H: aggregate SRAM capacity (word problem)

Step 1 — Sum per-core SRAM. Why this step? The parent note's key correction: capacity is the sum over cores, never per-core. Each core is tiny; the mesh is vast.

$S_{t o t} = 850, 000 \times 48 kB = 4.08 \times 1 0^{7} kB$

Step 2 — Convert to GB. Why this step? The model's requirement is in GB, so we must match units. Using the decimal convention pinned at the top of this page, $1 GB = 1 0^{6} kB$ .

$S_{t o t} = \frac{4.08 \times 1 0 ^{7}}{1 0 ^{6}} GB = 40.8 GB$

Step 3 — Compare to the model. Why this step? Fit means capacity $\geq$ requirement.

$40.8 GB \geq 30 GB ✓ — it fits, with 40.8 - 30 = 10.8 GB spare.$

So the answer to (a) is $S_{t o t} = 40.8$ GB and to (b) yes, the model fits entirely on-wafer with 10.8 GB of headroom — no external DRAM needed.

Verify: $850, 000 \times 48 = 40, 800, 000$ kB $= 40.8$ GB ✓; $40.8 - 30 = 10.8$ GB headroom ✓.

Example 7 — Cell I: sparsity speedup

Step 1 — Work remaining = nonzero fraction. Why this step? Recall $z$ is the fraction of activations that are zero. Only nonzero activations trigger a multiply, so useful work scales with $(1 - z)$ — Sparsity in Neural Networks.

$work fraction = 1 - z = 1 - 0.75 = 0.25$

Step 2 — Speedup = inverse of work done. Why this step? If you do only a quarter of the operations at the same rate, you finish in a quarter of the time.

$speedup = \frac{1}{1 - z} = \frac{1}{0.25} = 4 \times$

Step 3 — Edge check $z \to 1$ . Why this step? All-zero → infinite speedup (nothing to do), the sensible limit; $z \to 0$ (dense) → speedup $1$ (no benefit), which is why dense GPUs "gain nothing" here.

Verify: $1/0.25 = 4$ ✓; limits $z = 0 \Rightarrow 1 \times$ , $z \to 1 \Rightarrow \infty$ both monotonic. ✓

Example 8 — Cell J: the combined exam twist

Step 1 — Surviving cores. Why this step? Everything downstream (memory, bandwidth) depends on how many cores are alive after defects. Here $f$ is the fraction that fail, so $(1 - f)$ survive.

$M_{g oo d} = M (1 - f) = 1, 000, 000 \times 0.88 = 880, 000$ $880, 000 \geq 850, 000 ✓ (core constraint met)$

Step 2 — Surviving SRAM. Why this step? Capacity is summed over good cores only — dead cores contribute nothing. Decimal prefixes as pinned at the top.

$S_{t o t} = 880, 000 \times 48 kB = 4.224 \times 1 0^{7} kB = 42.24 GB$ $42.24 GB \geq 38 GB ✓ (memory constraint met)$

Step 3 — Aggregate bandwidth of survivors. Why this step? Same additive-link logic as Example 4, but only live cores count.

$B_{t o t} = 880, 000 \times 1 GB/s = 880, 000 GB/s = 880 TB/s$

Step 4 — Verdict. All three constraints pass; the tightest margin is memory ( $42.24$ vs $38$ GB, only $4.24$ GB spare). Why note this? In an exam, "which margin is thinnest" is the follow-up question — memory is the binding constraint here.

Verify: $1 0^{6} \times 0.88 = 880, 000$ ✓; $880, 000 \times 48 = 42, 240, 000$ kB $= 42.24$ GB ✓; $42.24 \geq 38$ ✓; $880, 000$ GB/s $= 880$ TB/s ✓.

Recall One-line recap of the matrix

Yield collapses exponentially with area (Ex. 1–2), so you over-provision cores by $1/ (1 - f)$ (Ex. 3), then the survivors deliver additive bandwidth (Ex. 4, 8), 1-ns local / µs-worst latency (Ex. 5), summed SRAM capacity (Ex. 6, 8), and sparsity multiplies throughput (Ex. 7).

Related tooling context: Chiplets & 2.5D/3D Integration · Liquid Cooling & Power Delivery Networks · Hinglish version: 6.5.17 Wafer-scale engines (Cerebras-style) (Hinglish).