Noesis
5.3.14 · D3Advanced Microarchitecture

Worked examples — Simultaneous multithreading (SMT - hyperthreading)

3,014 words14 min read↑ Back to topic
🇮🇳 Yeh page Hinglish mein padho →

This page is the hands-on companion to the SMT topic note. There we explained why a core wastes cycles; here we compute exactly how much SMT wins (or loses) in every kind of workload you could throw at it. If a term like IPC or utilization feels new, we rebuild it below before using it.

The scenario matrix

Every SMT situation is one cell of this grid. The overlap column is the key: how much of one thread's idle time the other thread can usefully fill.

Cell Thread A profile Thread B profile What limits speedup Expected result
C1 CPU-bound memory-bound complementary → big win ~1.5×
C2 CPU-bound CPU-bound fight for same ALUs ~1.1× (tiny)
C3 memory-bound memory-bound shared DRAM bandwidth ~1.3× (bandwidth-capped)
C4 one thread only (idle sibling) degenerate: SMT = off exactly 1.0×
C5 any any, but 100% idle sibling limiting / ideal case up to 2.0× ceiling
C6 mix, tunable — Amdahl formula sweep
C7 word problem (web server) — real-world throughput compute requests/sec
C8 exam twist: SMT hurts — contention > overlap speedup < 1.0

We now walk one worked example per cell (C6 gets two points on its curve).

C1 — Complementary threads (the flagship win)

Forecast: guess a number between 1.0 and 2.0 before reading on. Complementary → expect a strong win but nowhere near 2×.

  1. Sequential (no SMT) time. cycles, cycles. Total cycles. Why this step? Without SMT the core runs one thread to completion, then the other — so times add.
  2. SMT time per thread. , cycles. Why this step? Under SMT both run at the same time, each at its own contended IPC.
  3. When do both finish? They overlap, so wall-clock cycles. Why this step? The slower thread is the "long pole"; the faster one hides entirely inside it — its whole runtime is buried in B's window, so it adds nothing to wall-clock.
  4. Speedup. . Why this step? Speedup is old wall-clock ÷ new wall-clock. We did the same total work (2000 insns) in less time, and that ratio is exactly what "how much faster" means.

Look at the figure: the amber blocks (thread A's work) slot neatly into the white gaps (thread B's stalls).

Figure — Simultaneous multithreading (SMT - hyperthreading)

Verify: a tempting shortcut is "combined IPC , so time cycles." This is wrong because that formula assumes both threads run for the whole window at their combined rate — but thread A actually stops at 555.6 cycles, after which only B is running at IPC . So the combined rate only holds for the first 555.6 cycles, not all the way. The correct answer comes from the finishing times directly: B finishes last at 1666.7 cycles, and A already finished at 555.6, so both are done at 1666.7 cycles. Speedup . Units: cycles/cycles = dimensionless. ✓

C2 — Two CPU-bound threads (they collide)

Forecast: almost no idle units to fill — expect barely above 1×.

  1. Sequential time. Each cycles; total cycles. Why this step? Same as C1: run one thread to completion, then the other, so the two runtimes add.
  2. SMT time per thread. cycles. Both overlap ⇒ wall-clock . Why this step? The two threads are symmetric (identical IPC), so they finish together; the wall-clock is just one thread's SMT runtime.
  3. Speedup. . Why this step? Old wall-clock ÷ new wall-clock, exactly as in C1. Small gain because most units were already busy.

Verify: combined IPC under SMT , which is above the single-thread IPC — so total throughput really did rise, consistent with a speedup above 1. ✓ Utilization rose from to — only 10 points of idle units were recoverable, which is why the gain is tiny. ✓

C3 — Two memory-bound threads (a bandwidth model)

Forecast: the units are wide open, but the pipe to memory is narrow. Guess a gain, but a small one — well under 2×.

  1. Sequential time. Each cycles; total cycles. Why this step? One thread at a time; a lone thread only reaches IPC , so each takes 2000 cycles and they add.
  2. What IPC does each thread get under SMT? The two threads want combined, but the pipe only delivers . So the memory system throttles them to a combined , split evenly: each thread gets . Why this step? This is the whole point of C3 — the ceiling is , not . When demand () exceeds supply (), each thread is scaled down to its fair share.
  3. SMT time per thread. cycles. Both overlap ⇒ wall-clock cycles. Why this step? Symmetric threads finish together, so wall-clock is one thread's SMT runtime.
  4. Speedup. . Why this step? Old wall-clock ÷ new wall-clock. Note the gain comes purely from doing two threads' work in one window; even throttled, running both at once beats running them one after another.

Verify: the honest ceiling check — combined SMT IPC exactly, so we are exactly bandwidth-saturated and never above it. ✓ Compare with the naive "each stays at " fantasy: that would demand combined , which the pipe cannot deliver, so the throttle to is forced. The execution units sit at utilization — nearly empty — proving the bottleneck is memory bandwidth, not units. ✓ Speedup , below 2× as forecast. ✓

C4 — Degenerate case: only one thread exists

Forecast: with nobody to fill the bubbles, SMT should do... nothing.

  1. SMT time. With no B instructions ever ready, A runs exactly as alone: cycles. Why this step? The scheduler picks from a ready queue; if only A has ready ops, A gets every slot — the presence of an empty second context changes nothing.
  2. Speedup vs single-thread. . Why this step? Old wall-clock ÷ new wall-clock, and the two are identical, so the ratio is exactly 1 — this is the baseline that proves SMT is a no-op when there is nothing to overlap.

Verify: duplicated per-thread state (PC, registers) sits unused; zero performance cost when idle because the shared execution units are still 100% available to A. Speedup exactly . ✓ This is the correct "SMT never hurts a lone thread's throughput" baseline.

C5 — Limiting / ideal case (the 2× ceiling)

Forecast: the theoretical best — guess exactly 2×.

  1. Sequential time. Each ; total cycles. Why this step? One thread then the other; runtimes add.
  2. SMT time. No contention ⇒ each stays IPC ⇒ each cycles, overlapping ⇒ wall-clock . Why this step? Because A only touches integer units and B only touches FP units, they never compete, so neither slows down — the ideal that no other cell reaches.
  3. Speedup. . Why this step? Old wall-clock ÷ new wall-clock. We did two threads' work in the time of one, and with zero slowdown that ratio hits the hard ceiling of 2 for a 2-way core.

The figure below is a resource map, not a timeline. The vertical axis lists the 8 execution units (unit 0 at the bottom, unit 7 at the top). The amber blocks are the 4 integer units claimed by thread A; the cyan blocks are the 4 FP units claimed by thread B. The key visual takeaway: the two colours never share a row, so combined demand ( units of A of B ) fits inside with no overlap — that geometric non-collision is exactly why the 2× ceiling is reachable here.

Figure — Simultaneous multithreading (SMT - hyperthreading)

Verify: combined IPC , and , so no execution-unit conflict — the ceiling is reachable only because units are abundant and workloads are disjoint. . This is the absolute upper bound for 2-way SMT.

C6 — Amdahl sweep (formula check, two points)

  1. At : denominator . Speedup . Why this step? 60% of cycles offer half-fillable bubbles, so we replace of the work with effective time.
  2. At : denominator . Speedup . Why this step? Nearly everything is stall time ⇒ approaching the limit of .

The plot below has (the memory-bound fraction, 0 to 1) on the horizontal axis and speedup on the vertical axis. The cyan curve is ; the two amber dots mark our worked points and ; the dashed white line is the ceiling. Visual takeaway: the curve rises from 1.0 (no stalls, no help) toward 2.0 (all stalls, sibling fills half) and flattens as it approaches the ceiling — showing diminishing returns as workloads become more memory-bound.

Figure — Simultaneous multithreading (SMT - hyperthreading)

Verify: limits — at (all CPU-bound) speedup ✓; at speedup ✓; monotonic increasing between them ✓. Our two points and lie on that curve. ✓

C7 — Real-world word problem (a web server)

Forecast: SMT multiplies throughput by the C6 factor .

  1. Cycles per request (single thread). cycles. Why this step? Instructions ÷ IPC = cycles; this converts "work per request" into "time per request."
  2. Requests/sec, no SMT. req/s. Why this step? Cycles-per-second ÷ cycles-per-request = requests-per-second; the clock rate sets how many request-sized time-slices fit in a second.
  3. Apply SMT speedup (from C6, ): req/s. Why this step? Because this workload's stall fraction is exactly the we swept in C6, its aggregate throughput rises by that same Amdahl factor — this is where the abstract curve becomes a concrete "requests per second," reusing the CPU-vs-memory mix idea directly.

Verify: cross-check via effective IPC. Speedup means aggregate IPC ; cycles per request effectively ; req/s ✓ — matches step 3. Units: (cycles/s)/(cycles/req) = req/s ✓. Wrap-up: the server now serves about 43% more requests per second for a ~5% die-area cost, and the single web request itself is no faster — throughput up, per-request latency flat-to-slightly-up, exactly the SMT trade the topic note promised.

C8 — Exam twist: when SMT hurts

Forecast: if per-thread IPC crashes hard enough, aggregate throughput can fall below running them one at a time. Guess: speedup < 1.0?

  1. Sequential time. Each ; total cycles. Why this step? One thread then the other; runtimes add — this is the baseline SMT must beat.
  2. SMT time. Each ; overlap ⇒ wall-clock . Why this step? Symmetric threads finish together, so wall-clock is one thread's SMT runtime; the catch is that each thread's IPC collapsed from 2.0 to 0.8 because they evict each other's cache lines.
  3. Speedup. . Why this step? Old wall-clock ÷ new wall-clock — and here the ratio is below 1, meaning SMT made total throughput worse, the counter-intuitive result this cell exists to show.

Verify: combined IPC under SMT single-thread — so total throughput actually dropped. Speedup ✓. Real cause: shared cache capacity halved per thread + predictor interference. This is why SMT is a BIOS/OS toggle: for cache-thrashing workloads you turn it off.

Recall Self-test: match cell to outcome

Which cell gives exactly 1.0× and why? ::: C4 — a lone thread with an idle sibling; nothing to fill bubbles, no contention, so SMT is a no-op. Which cell gives exactly 2.0× and why? ::: C5 — disjoint resource use on a wide core; combined IPC stays under , so zero contention hits the theoretical ceiling. Which cell gives < 1.0× and why? ::: C8 — cache thrashing crushes per-thread IPC so far that aggregate throughput falls below sequential. At , what speedup does Amdahl predict? ::: .

Connections