Multicore vs manycore designs: Exercises — Hardware deep dive

This page is a self-testing ladder. Each rung is harder than the last. Cover the solution, try the problem, then reveal. This page stands alone — every symbol it uses is defined right here before it appears.

Definition The five symbols used on this whole page

Before any equation, meet the cast. Each is a plain-English quantity, not a mystery letter.

$P$ = the parallel fraction: what slice of the program's runtime can be split across cores. A pure number from $0$ (nothing parallel) to $1$ (everything parallel). Example: "70% serial" means $P = 0.30$ .
$N$ = the core count: how many cores share the work. A whole number ( $8$ , $512$ , $6912$ , ...).
$f$ = the clock frequency: how many cycles each core ticks per second, in GHz ( $1 0^{9}$ cycles/sec). A "4 GHz" core does 4 billion ticks a second.
$IPC$ = instructions per cycle: how many instructions one core finishes on one tick. A fast out-of-order core does $\approx 4$ ; a simple in-order core does $\approx 1$ .
$U$ = utilization: the fraction ( $0$ to $1$ ) of cores actually doing useful work at any moment, after subtracting stalls and idle threads. $U = 0.9$ means 90% busy.

Figure s01 — the map every problem lives on. Read this picture first and return to it often. The horizontal axis is core count $N$ (log scale, so $8$ and $512$ can share one plot), and the vertical axis is aggregate throughput in GIPS (billions of instructions/sec for the whole chip — higher is better). Three points are marked:

The orange dot (low-left) is the multicore: only $8$ cores, so it sits far left, and its total throughput ( $115.2$ GIPS) is modest — but each of those cores is a sprinter, fastest on a single thread.
The teal dot (far right, high) is the manycore: $512$ cores push it right and lift its aggregate throughput to $537.6$ GIPS — yet each core is a slow walker.
The plum diamond (middle) is a heterogeneous chip: a few fast cores plus many slow ones, deliberately sitting between the two extremes. The whole page asks where on this map does my workload want to live? — L2.3 computes and places the two labelled dots, L3 finds the crossover between them, L4 ties position to transistor and power budgets, and L5 builds the plum diamond in the middle.

Level 1 — Recognition

Can you name the philosophy and read the definitions?

Recall Solution L1.1

Manycore. The deciding features:

Huge core count (6912, far above the 2–16 range of multicore).
Simple in-order cores with 1 IPC and tiny caches / no L2 — the parent's manycore checklist exactly. This is the NVIDIA A100 profile from the parent's Example 2. On the map in Figure s01, this chip is the teal dot on the far right.

Recall Solution L1.2

Cores (adding more cores). The transistor budget is fixed, so complexity per core and number of cores are in direct competition. This is the central sentence of the parent note.

Recall Solution L1.3

(a) → Manycore. Fully data-parallel, every output element independent.
(b) → Multicore. Pointer chasing = unpredictable branches + latency-sensitive, terrible for simple cores.
(c) → Multicore. A large serial fraction means fast individual cores matter far more than core count.

Level 2 — Application

Can you plug numbers into the two models?

Recall Solution L2.1

Step 1 — pick the tool. Why Amdahl and not throughput? The question asks how much faster one job finishes, which is a speedup (latency) question — that is exactly what Amdahl's Law measures. Throughput would answer a different question (total work/sec). Step 2 — plug in. Why these numbers where? $P = 0.30$ is the parallel slice, $1 - P = 0.70$ is the un-shrinkable serial slice, and $N = 8$ is how many cores split the parallel slice: $Speedup = \frac{1}{( 1 - 0.30 ) + \frac{0.30}{8}} = \frac{1}{0.70 + 0.0375} = \frac{1}{0.7375} \approx 1.36 \times$ What this means: even doubling to 16 cores barely helps — the $0.70$ serial term dominates. This is why multicore keeps each core fast rather than adding cores.

Recall Solution L2.2

Step 1 — set the serial term. Why does it vanish? $P = 1$ means nothing is serial, so $1 - P = 0$ : there is no un-shrinkable part to strangle the speedup. Step 2 — divide the whole job by $N$ . Why is the denominator just $1/ N$ ? With the serial term gone, the entire runtime is the parallel part $P = 1$ split across all $N = 6912$ cores: $Speedup = \frac{1}{0 + \frac{1}{6912}} = 6912 \times$ What this looks like: perfect parallelism makes speedup grow linearly with $N$ — the whole reason manycore chips pile on cores. (Real hardware hits ~4000× because memory bandwidth, not cores, becomes the limit.)

Recall Solution L2.3

Step 1 — pick throughput, not speedup. Why? The question asks how much total work per second the whole chip does across all threads — that is throughput, so we multiply the four factors from the definition box. Aggregate throughput multiplies all four factors: $Multicore = 8 \times 4 \times 4 \times 0.9 = 115.2 GIPS$ $Manycore = 512 \times 1.5 \times 1 \times 0.7 = 537.6 GIPS$ Ratio: $537.6/115.2 = 4.67 \times$ . Manycore wins throughput by ~4.7× — this places the two dots on Figure s01 (orange multicore low, teal manycore high). Step 2 — single-thread speed: keep the units straight. Why does $N$ drop out? A single thread runs on one core, so core count is irrelevant to it — only the per-core factors survive.

Work per cycle per core is just $IPC$ : multicore $4$ vs manycore $1$ ⇒ 4× more work each clock.
Throughput per core is $f \times IPC$ : multicore $4 \times 4 = 16$ GIPS/core vs manycore $1.5 \times 1 = 1.5$ GIPS/core ⇒ $16/1.5 \approx 10.7 \times$ faster per core once the clock is included. So the manycore wins aggregate work by 4.7× yet loses single-thread work by 4× per cycle (≈10.7× per second). That opposite pull is exactly the two-axis tension in Figure s01.

Level 3 — Analysis

Can you find break-even points and reason about limits?

Recall Solution L3.1

Step 1 — find the ceiling. Why look at $N \to \infty$ ? Because it isolates the part of the runtime that no amount of parallelism can touch — the serial fraction. As $N \to \infty$ , the parallel term $P / N \to 0$ (dividing a fixed amount of work across ever more cores shrinks it toward nothing), leaving only $(1 - P)$ : $Speedup_{m a x} = \frac{1}{1 - P} = \frac{1}{0.05} = 20 \times$ Step 2 — set the 90% target. Why 90% of 20, i.e. 18? We want the core count where we've captured most of the achievable gain, so we ask "which $N$ gives speedup $= 18$ ?" Set the full formula equal to 18: $18 = \frac{1}{0.05 + \frac{0.95}{N}}$ Step 3 — invert. Why invert? The unknown $N$ is trapped inside the denominator; flipping both sides frees it. Taking reciprocals of both sides: $0.05 + \frac{0.95}{N} = \frac{1}{18} = 0.0556$ Step 4 — isolate $N$ . Subtract the serial term, then divide: $\frac{0.95}{N} = 0.0556 - 0.05 = 0.00556 \Rightarrow N = \frac{0.95}{0.00556} \approx 171$ So ~171 cores to reach 90% of the 20× limit. Insight: past that, extra cores are nearly wasted — the serial 5% is the wall.

Recall Solution L3.2

Set the two Amdahl speedups equal: $\frac{1}{( 1 - P ) + \frac{P}{8}} = \frac{1}{( 1 - P ) + \frac{P}{512}}$ The numerators match, so equality requires $\frac{P}{8} = \frac{P}{512}$ , which only holds at $P = 0$ . For any $P > 0$ , the manycore denominator is smaller (since $P /512 < P /8$ ), so manycore's Amdahl speedup is always ≥ multicore's for every $P > 0$ , and strictly greater whenever $P > 0$ . The catch: this pure-Amdahl view ignores per-core speed. Once you fold in that each manycore core is much slower per clock (from L2.3), the multicore wins for small $P$ — which is exactly L3.3.

Recall Solution L3.3

Step 1 — build the score. Why multiply per-core work by speedup? Amdahl speedup is relative to each chip's own baseline, so it hides that a multicore core is intrinsically 16× stronger. Multiplying by the per-core weight $w$ puts both chips on one common absolute yardstick: $score = w \cdot \frac{1}{( 1 - P ) + P / N}$

Multicore: $w_{A} = 16$ , $N_{A} = 8$ .
Manycore: $w_{B} = 1$ , $N_{B} = 512$ . Step 2 — set them equal. Why equal? The crossover is the exact $P$ where neither wins — solve for that and you know which side of it each chip dominates: $\frac{16}{( 1 - P ) + \frac{P}{8}} = \frac{1}{( 1 - P ) + \frac{P}{512}}$ Step 3 — cross-multiply. Why? It clears both fractions in one move, turning the equation into plain polynomial arithmetic: $16 [(1 - P) + \frac{P}{512}] = (1 - P) + \frac{P}{8}$ Step 4 — expand both sides. Distribute and convert the fractions to decimals ( $16/512 = 0.03125$ , $1/8 = 0.125$ ): $16 - 16 P + 0.03125 P = 1 - P + 0.125 P$ Step 5 — collect like terms. Gather constants on the left, $P$ -terms on the right: $16 - 15.96875 P = 1 - 0.875 P \Rightarrow 15 = 15.96875 P - 0.875 P = 15.09375 P$ Step 6 — solve. $P = \frac{15}{15.09375} \approx 0.9938$ So the manycore only overtakes when $P \approx 0.994$ — you need over 99% parallel code before the swarm of slow cores beats the few fast ones. That single number is the whole story of the chapter.

Level 4 — Synthesis

Can you combine the transistor budget with performance?

Recall Solution L4.1

$N_{complex} = \frac{400}{10} = 40, N_{simple} = \frac{400}{0.2} = 2000$ Ratio $= 2000/40 = 50$ . You trade 1 complex core for 50 simple cores — matching the parent's tradeoff derivation.

Recall Solution L4.2

Absolute score $= w \cdot \frac{1}{( 1 - P ) + P / N}$ with $P = 0.98$ . Complex (w=16, N=40): $denom = 0.02 + \frac{0.98}{40} = 0.02 + 0.0245 = 0.0445$ $score_{C} = 16/0.0445 = 359.6$ Simple (w=1, N=2000): $denom = 0.02 + \frac{0.98}{2000} = 0.02 + 0.00049 = 0.02049$ $score_{S} = 1/0.02049 = 48.8$ Complex layout wins ( $359.6$ vs $48.8$ , about $7.4 \times$ ). Even at 98% parallel, the 2% serial part plus the 16× per-core weight keeps the fast cores ahead. It takes ~99.4% (L3.3 logic) to flip.

Recall Solution L4.3

Complex: $40 \times 30 = 1200$ W — way over 200 W. Max cores that fit: $⌊ 200/30 ⌋ = 6$ cores. Drop $40 - 6 = 34$ .
Simple: $2000 \times 0.4 = 800$ W — over 200 W. Max: $⌊ 200/0.4 ⌋ = 500$ cores. Drop $2000 - 500 = 1500$ . Lesson: area is not the real limit anymore — power is. The die fits far more cores than you can afford to switch on. This is the "dark silicon" reality behind Heterogeneous Computing.

Level 5 — Mastery

Can you design the tradeoff yourself?

Recall Solution L5.1

Normalization convention (define $T$ first). Let $T$ be the runtime of the entire job on a single complex core running at per-core work 1 — a reference stopwatch we set to $T = 1$ . Splitting by the given percentages: the serial control loop takes $0.6$ of that unit and the parallel kernel takes $0.4$ . Every runtime below is measured in these same units, so smaller = faster. Strategy: put the serial loop on one fast complex core, and spend the rest of the power budget on simple cores for the parallel kernel — this is the Heterogeneous Computing idea. Reserve 1 complex core: 30 W. Remaining power $= 200 - 30 = 170$ W. Simple cores: $⌊ 170/0.4 ⌋ = 425$ cores.

Heterogeneous chip: serial runs on the complex core, 16× faster ⇒ $0.6/16 = 0.0375$ . Parallel runs on 425 simple cores (each 1× work) ⇒ $0.4/425 = 0.000941$ . Total $\approx 0.0375 + 0.000941 = 0.03844$ .
Pure complex, 6 cores: serial $0.6/16 = 0.0375$ . Parallel on 6 complex cores (each 16× work, 6 in parallel) ⇒ $0.4/ (16 \times 6) = 0.004167$ . Total $\approx 0.0375 + 0.004167 = 0.04167$ . Speedup of heterogeneous vs pure-complex $= 0.04167/0.03844 \approx 1.084$ , i.e. ~8.4% faster, while both obey the same 200 W cap. The synthesis insight: the serial loop needs one fast core (16× beats 425 slow cores at serial work — since simple cores give no serial speedup), while the parallel kernel rewards the swarm. Mixing both is why real chips (Heterogeneous Computing, big.LITTLE, CPU+GPU) beat any single philosophy.

Recall Solution L5.2

A manycore core hides long memory stalls with thread-level parallelism: when one thread stalls waiting on memory (100+ cycles), the hardware scheduler instantly swaps in another ready thread, so the pipeline stays busy without needing a big cache to avoid the stall or a branch predictor to guess around it. Because there are always more threads waiting, latency is tolerated rather than eliminated, which frees the transistor budget to be spent on yet more cores instead of on a deep Memory Hierarchy. A multicore core faces the opposite situation: it often runs a single latency-critical thread with no spare threads to swap in, so a memory stall or a mispredicted branch stalls the whole job. It therefore must invest in a large Memory Hierarchy (big private L1/L2) to keep data close and in branch prediction to keep the pipeline full. In one line: many threads let you tolerate latency, so you can strip the core bare; few threads force you to eliminate latency, so you must pay for caches and prediction — which is precisely why the two philosophies split the transistor budget in opposite directions.

Recall Self-test summary

Which single number captures "how parallel must code be before manycore beats multicore per-die"? ::: About $P \approx 0.994$ — over 99% parallel (L3.3). What binds core count in modern chips, area or power? ::: Power (L4.3) — the die fits far more cores than the power cap allows. Why can manycore skip branch prediction? ::: Thread-level parallelism hides stalls; a stalled thread is swapped for a ready one (L5.2).