6.1.4 · D5Parallelism & Multicore
Question bank — Multicore vs manycore designs
This is a misconception-hunting page for the parent topic. No heavy arithmetic here — every question below targets a belief that feels true but breaks under scrutiny. Read the prompt, say your answer out loud, then reveal.
The vocabulary you need first
Before any trap makes sense, we must earn every symbol and acronym it uses. Nothing below is assumed.
- Latency = how long one task takes from start to finish (a stopwatch on a single job).
- Throughput = how many tasks finish per second (a counter watching the whole factory).
A multicore chip chases low latency; a manycore chip chases high throughput. Almost every trap on this page is really about confusing these two.
Look at the figure. On the left, one fat pipe moves a single ball fast — that's low latency, the multicore ideal. On the right, many thin pipes each move a ball slowly, but so many at once that more balls exit per second — that's high throughput, the manycore ideal. Same total "pipe material" (transistors), spent two opposite ways.
Two more acronyms show up in the "why" section, so we define them now, not later:
Two pieces of hardware get blamed in the area-budget traps, so meet them now:
Amdahl's formula, written out in full
The single equation this whole page leans on is Amdahl's Law. We state it here so you never have to click away:
The graph shows why the serial fraction is "the killer". Each curve fixes and lets (cores) grow along the horizontal axis. Notice every curve flattens into a ceiling: as the term vanishes, leaving . Even (top curve) can never pass . Adding cores past the flat part buys nothing — that is the trap half these questions spring.
True or false — justify
More cores always means more speedup
False — with fixed serial fraction , Amdahl's Law flattens to a ceiling of ; if 30% of a job is serial () then even infinite cores give at most .
A manycore chip is always faster than a multicore chip
False — only for regular data-parallel work; on a single branchy thread the multicore core (more instructions finished per clock, higher clock) finishes each instruction faster, so it wins by a wide margin.
Each manycore core is faster than each multicore core because there are more of them
False — the opposite: individual manycore cores are simple, in-order, and low-clock, finishing only 1–2 instructions per cycle. Aggregate throughput is high; per-core latency is worse.
Doubling the core count doubles the throughput
Only if utilization stays fixed — but as cores grow they contend for one shared memory bus, so each waits longer for data (Memory Hierarchy), and Cache Coherence messages between cores multiply; both drop utilization (more idle time), so throughput grows less than .
A multicore CPU can be turned into a manycore CPU just by shrinking the caches
False — walk the area budget: in a big core the cache is only part of the bulk; the out-of-order scheduler and branch predictor are the other large blocks. Freeing just the cache leaves those in place, so you don't recover nearly enough area to fit tens of simple cores. You must strip all three.
Manycore designs need branch prediction the least
True — they hide stalls by switching to another ready thread instead of guessing which way a branch goes; with thousands of threads available, a mispredicted branch just parks that one thread while others keep the ALUs busy.
The transistor budget is a soft constraint you can exceed with a better design
False — die area is fixed by the process node and cost target; every mm² spent on control logic is a mm² not spent on cores. It's a hard, conserved tradeoff.
A single-threaded program runs faster on a 512-core manycore chip than an 8-core multicore chip
False — a single thread uses one core ( effectively); the manycore's one weak core loses badly to the multicore's one strong core. The other 511 idle cores can't help one thread.
Hyper-Threading gives you twice the cores
False — it lets one physical core hold two thread contexts to hide stalls; it raises utilization of existing hardware, but (real cores) is unchanged.
Spot the error
"Amdahl's speedup formula proves manycore chips are pointless, since serial code caps everyone."
The error is scope — Amdahl bounds a single fixed-size job through its term; manycore shines on jobs that are (near) 100% parallel () and on running many independent jobs at once, where the per-job serial cap barely applies.
"We picked GPUs for our database because they have thousands of cores."
Wrong workload — database queries are pointer-chasing, branchy, and full of locks (sequential critical sections); those punish simple manycore pipelines. This is a classic multicore-wins case.
"Manycore cores waste power because there are so many of them."
Each simple core draws only ~0.1–1 W versus 15–50 W for a complex core; per-useful-operation, manycore is often more energy-efficient — see Power Consumption. Count matters less than per-op cost.
"Since gives max speedup, an 8-core chip already wastes 12 cores."
The is the asymptotic ceiling (the flat part of the curve, at ); plug into and you get about , well below 20 — so those 8 cores are all doing useful work. Don't confuse the limit with the current value.
"Adding L3 cache to a manycore chip would make it a multicore chip."
A shared cache is one feature, not the defining line. The line is few strong cores vs many weak cores; a manycore chip with cache is still manycore.
"A task that's 100% parallel will get exactly speedup on cores."
Only ideally — the formula gives only when utilization is perfect (ALUs never idle); real speedup is capped by memory bandwidth, synchronization, and interconnect traffic, so the A100 example reaches ~4000× of a theoretical ~6912×.
"Thread switching is cheap, so multicore chips switch threads freely to hide memory stalls."
Reversed — on multicore, a switch costs thousands of cycles (software-managed), so it can't hide a 100-cycle stall; manycore switches in ~1 cycle in hardware, which is exactly how it hides stalls.
Why questions
Why can't we build a chip that's both maximally fast per-thread AND maximally parallel?
Fixed transistor/area budget — the out-of-order scheduler, big caches, and branch predictor that make a core fast are exactly the area you'd need to add more cores. You spend the budget one way or the other.
Why does manycore hide memory latency with threads instead of caches?
With hundreds of threads ready, a stalled thread is simply swapped out (cheap in hardware) while others compute; this keeps the ALUs (the calculating circuits) busy without spending scarce area on huge caches — see Memory Hierarchy.
Why do GPUs favour "same instruction, many threads" execution?
That is SIMT — sharing one instruction stream across a group of threads lets one control unit drive many ALUs, so almost all the transistors go to arithmetic — perfect for data-parallel work. See SIMD vs MIMD.
Why is single-thread performance still worth chasing even on an 8-core machine?
The serial fraction runs on one core and, by Amdahl, dominates total time as parallelism grows; a fast serial core shrinks the part that no amount of Thread-Level Parallelism can touch.
Why does branch-heavy code punish manycore designs specifically?
In SIMT, threads in a group must follow the same instruction; when branches send threads down different paths (divergence), the hardware runs each path serially, wasting the ALUs meant to run in lockstep.
Why do heterogeneous chips pair a few big cores with many small cores?
To match each workload to the right engine — big cores for latency-critical serial code, small cores/accelerators for throughput-heavy parallel code — instead of forcing one style to do both, the idea behind Heterogeneous Computing.
Why does the serial fraction get called "the killer"?
Because it sets the hard ceiling — as the term vanishes and speedup approaches ; no amount of parallel hardware removes the serial time, so a tiny serial slice quietly caps everything.
Edge cases
What is the speedup when (nothing parallel), on any number of cores?
Exactly — Amdahl gives ; extra cores sit idle, so multicore or manycore makes no difference.
What is the speedup when (fully parallel), on cores?
Exactly ideally — ; this is the only case where more cores scale linearly, and it's precisely the manycore sweet spot.
What happens to throughput as utilization approaches 0 (all threads stalled on memory)?
Throughput collapses toward zero regardless of core count — since throughput scales directly with utilization (the busy-fraction of the ALUs), a bandwidth-starved manycore chip behaves like it has almost no cores.
What if a job has more independent threads than the chip has cores?
Manycore tolerates this well — extra threads queue and instantly fill in when others stall, raising utilization; multicore must time-slice them in software at high switching cost.
What if a job has far fewer threads than cores (say 2 threads on 512 cores)?
510 cores go idle — throughput hardware is wasted (utilization near zero), and the two threads would have been better served by a fast multicore core each. Wrong tool for the job.
What is the "speedup" for a single-thread program on any parallel chip?
on the cores axis — it uses one core (); the only wins come from that core's own clock and instruction throughput, not from parallelism, which is why multicore keeps each core strong.
As the number of simple cores grows without bound, what limits real speedup instead of Amdahl?
Physical limits take over — memory bandwidth, interconnect/network-on-chip traffic, and coherence overhead (Cache Coherence) flatten the curve long before the theoretical linear line does.
Recall Quick self-test
The single sentence that unlocks half these traps ::: Multicore optimises latency (one task fast); manycore optimises throughput (many tasks per second) — most "wrong intuitions" swap those two goals. The one law that caps every parallel speedup ::: Amdahl's Law, — the serial fraction sets the ceiling no matter how many cores you add. What P, N and utilization stand for ::: = parallel fraction (0–1), = number of cores, utilization = fraction of time the ALUs are actually working instead of idling.