Multicore vs manycore designs: Visual walkthrough — Hardware deep dive

This page builds the central result of multicore vs manycore from absolute zero: the equation that decides whether you want a few fast cores or many slow cores. That equation is Amdahl's Law. We will not assume you have ever seen it. We will draw it into existence, one bar at a time.

Step 1 — Draw the work as a bar of time

WHAT. Forget cores for a moment. Take one program and let it run on one core. Measure how long it takes. Call that total time $T_{orig}$ — the length of the whole bar.

WHY. Speedup is always a comparison: new time versus old time. So we need a baseline first. Everything later is measured against this one bar.

PICTURE. The full bar below is $T_{orig}$ . We have not split anything yet — this is the "before" picture.

Step 2 — Split the bar into two honest pieces

WHAT. Colour the bar in two parts. One part is code that can run in parallel (many workers at once). The other part is code that must run one step at a time — no splitting allowed.

WHY. Not all code is equal. Adding a list of a million numbers? Split it — that is parallel. But "read the config file, THEN start the engine, THEN load the level" — each step needs the previous one done first. That is serial. The whole story hinges on this one split.

PICTURE. The red slice is the serial part — the part that will refuse to shrink no matter how many cores you buy. The black slice is parallel.

T_{orig} = cannot split T_{serial} + can split T_{parallel}

Step 3 — Turn the pieces into fractions (make it size-free)

WHAT. Instead of measuring in seconds, measure each piece as a fraction of the whole bar. Let $P$ be the parallel fraction. Then the serial fraction is whatever is left: $1 - P$ .

WHY. Seconds depend on the machine, the input, the weather. A fraction does not. Saying "90% of this program is parallel" ( $P = 0.9$ ) is a fact about the program itself, true on any hardware. This is why we divide out the units — the law should be about the code, not the clock.

PICTURE. Same bar, now labelled $0$ at the left and $1$ at the right. The red serial slice has width $1 - P$ ; the black parallel slice has width $P$ .

Step 4 — Hand the parallel slice to $N$ cores

WHAT. Now the machinery. We finally name the number of cores. Let == $N$ == be how many cores we give the program. Give it $N$ cores (see thread-level parallelism for how those cores get fed). The parallel slice gets shared across all $N$ — so it shrinks to $\frac{1}{N}$ of its old length. The serial slice does not shrink at all: one worker, one step at a time.

WHY. This is the whole point of parallel hardware: split the splittable work. If $N = 4$ cores each take a quarter of the parallel work, that piece finishes in a quarter of the time — and here is where the idealized "perfect load-balancing" assumption earns its keep: we assume each core gets an exactly equal quarter. But the serial piece has nobody to hand work to — it is stuck at full length.

PICTURE. Watch the black parallel slice collapse to $\frac{P}{N}$ while the red serial slice stays exactly the same width. The red bar is the villain of the whole story — stare at it.

Step 5 — Speedup gets a name: $S$ = old bar ÷ new bar

WHAT. We now define the quantity we have been chasing and give it a symbol. Let == $S$ == stand for speedup — how many times shorter the new bar is. Divide the old length by the new length.

WHY. "Twice as fast" literally means "half the time", i.e. old÷new $= 2$ . That is exactly the ratio we compute. Naming it $S$ lets us write it once and reason about it cleanly. It is a definition, not a trick.

PICTURE. Old bar (length $1$ ) sits on top of the shrunken new bar (length $(1 - P) + \frac{P}{N}$ ). The speedup $S$ is how many new bars fit into the old one.

Step 6 — Push $N$ to infinity: the wall appears

WHAT. Ask the extreme question: what if we had infinitely many cores? Let $N \to \infty$ . Then $\frac{P}{N} \to 0$ .

WHY. This is the single most important sentence in the whole topic. It tells us the best case ever, the ceiling no amount of hardware can beat. If the ceiling is low, buying more cores is pointless — and that is when manycore loses.

PICTURE. As $N$ grows, the parallel slice vanishes to a hair, but the red serial slice stays put. The bar cannot get shorter than the red block. That red block is the wall.

Step 7 — The speedup curve flattens (the picture that decides the war)

WHAT. Plot speedup $S$ on the vertical axis against number of cores $N$ on the horizontal axis, for a few values of $P$ .

WHY. A curve shows what a single number hides: diminishing returns. Early cores help a lot; later cores barely move the line. The shape is the argument.

PICTURE. Each curve rises then flattens into a horizontal ceiling at height $\frac{1}{1 - P}$ (the red dashed line for $P = 0.95$ ). High- $P$ workloads (matrix math, SIMD-friendly, near the top) keep climbing — those go to manycore GPUs. Low- $P$ workloads flatten early — those stay on multicore.

Step 8 — The degenerate corners (never leave a case unshown)

WHAT. Check the three extreme inputs so the formula never surprises you. Crucially, we do not invent new formulas for these — each corner is just a value of $P$ or $N$ plugged into the same $S = \frac{1}{( 1 - P ) + P / N}$ from Step 5.

WHY. A law you cannot sanity-check at its edges is a law you do not trust. Every corner must give an answer that feels right — and every one must fall out of the general expression, not a special rule.

PICTURE. Three mini-bars: fully serial (no shrink), fully parallel (shrinks to almost nothing), and single core (no change).

Case	Inputs	Plug into $S = \frac{1}{( 1 - P ) + P / N}$	Meaning
Fully serial	$P = 0$	$\frac{1}{( 1 - 0 ) + 0/ N} = \frac{1}{1} = 1$	No parallel part → cores do nothing. Speedup $= 1 \times$ .
Fully parallel	$P = 1$	$\frac{1}{0 + \frac{1}{N}} = N$	Perfect scaling → $N$ cores give $N \times$ . The GPU dream.
Single core	$N = 1$	$\frac{1}{( 1 - P ) + \frac{P}{1}} = \frac{1}{1} = 1$	One core = the baseline. Speedup $= 1 \times$ by definition. ✓

Notice all three answers came out of the one general formula — the extremes $P = 0$ and $P = 1$ are legal inputs, not exceptions.

The one-picture summary

Everything above — the bar, the split, the shrink, the ceiling — in a single frame.

Recall Feynman retelling — say it back in plain words

A program is a bar of time. Colour it: the red part must run one-step-at-a-time, the black part can be split. Give it $N$ cores — only the black part shrinks, to $1/ N$ of its size (assuming, ideally, perfect equal splitting and zero overhead); the red part refuses to budge. We measure everything relative to the original bar ( $T_{orig} = 1$ ), so speedup $S$ is just the old bar (=1) divided by the new, shrunken bar, which gives $\frac{1}{( 1 - P ) + P / N}$ . Now imagine more and more cores: $P / N$ becomes negligibly small, the black part disappears entirely, but the red part is still there — so the best you can ever do is $\frac{1}{1 - P}$ , set purely by how much red there is. Plot it and every curve flattens into that ceiling. If your code is mostly red (serial, branchy), the curve flattens instantly, so buy a few very fast cores — that's multicore. If your code is almost all black ( $P \to 1$ , like matrix math), the curve climbs forever, so buy thousands of tiny cores — that's manycore. The whole multicore-vs-manycore choice is just: how much red is in your bar?

Recall Quick self-test

With $P = 0.9$ and $N = 8$ , what is $S$ ? ::: $\frac{1}{0.1 + 0.9/8} = \frac{1}{0.2125} \approx 4.71 \times$ With $P = 0.9$ , what is the ceiling as $N \to \infty$ ? ::: $\frac{1}{1 - 0.9} = 10 \times$ Why does the red (serial) block never shrink? ::: Only one worker can run it, one step at a time — there is nothing to split, so $N$ never appears in the $(1 - P)$ term. Which curve shape screams "build a manycore GPU"? ::: One that keeps climbing (near $P = 1$ ), so every added core still earns real speedup. Name one assumption that makes real speedup worse than the formula. ::: Any of: imperfect load-balancing, thread-startup overhead, or communication/coherence cost — all assumed to be zero here.