Simultaneous multithreading (SMT - hyperthreading): Worked examples

5.3.14 · D3 · Hardware › Advanced Microarchitecture › Simultaneous multithreading (SMT - hyperthreading)

Yeh page SMT topic note ki hands-on companion hai. Wahan humne explain kiya tha ki ek core cycles kyun waste karta hai; yahan hum exactly calculate karte hain ki SMT kitna jeet ta (ya haarta) hai har tarah ke workload mein jo tum uske saath try kar sako. Agar IPC ya utilization jaisa koi term naya lagta hai, toh hum use neeche rebuild karte hain pehle, phir use karte hain.

Scenario matrix

Har SMT situation is grid ka ek cell hai. overlap column sabse key hai: ek thread ki idle time ka kitna hissa doosra thread usefully fill kar sakta hai.

Cell	Thread A profile	Thread B profile	Speedup kya limit karta hai	Expected result
C1	CPU-bound	memory-bound	complementary → bada win	~1.5×
C2	CPU-bound	CPU-bound	same ALUs ke liye fight	~1.1× (tiny)
C3	memory-bound	memory-bound	shared DRAM bandwidth	~1.3× (bandwidth-capped)
C4	sirf ek thread	(idle sibling)	degenerate: SMT = off	exactly 1.0×
C5	koi bhi	koi bhi, lekin 100% idle sibling	limiting / ideal case	up to 2.0× ceiling
C6	mix, $f$ tunable	—	Amdahl formula sweep	$\frac{1}{( 1 - f ) + f /2}$
C7	word problem (web server)	—	real-world throughput	requests/sec compute karo
C8	exam twist: SMT hurts	—	contention > overlap	speedup < 1.0

Ab hum har cell ke liye ek worked example chalate hain (C6 ko apne curve par do points milte hain).

C1 — Complementary threads (flagship win)

Forecast: padhne se pehle 1.0 aur 2.0 ke beech ek number guess karo. Complementary → ek strong win expect karo lekin 2× ke kahin bhi paas nahi.

Sequential (no SMT) time. $T_{A} = 1000/2.0 = 500$ cycles, $T_{B} = 1000/0.5 = 2000$ cycles. Total $= 500 + 2000 = 2500$ cycles. Yeh step kyun? SMT ke bina core ek thread ko completion tak run karta hai, phir doosre ko — toh times add hote hain.
SMT time per thread. $T_{A}^{'} = 1000/1.8 = 555.6$ , $T_{B}^{'} = 1000/0.6 = 1666.7$ cycles. Yeh step kyun? SMT ke under dono ek saath run karte hain, har ek apne contended IPC par.
Dono kab finish karte hain? Woh overlap karte hain, toh wall-clock $= max (555.6, 1666.7) = 1666.7$ cycles. Yeh step kyun? Slower thread "long pole" hai; faster thread poori tarah andar chhup jaata hai — uska pura runtime B ki window mein dab jaata hai, toh wall-clock mein kuch add nahi hota.
Speedup. $2500/1666.7 = 1.50$ . Yeh step kyun? Speedup hai purana wall-clock ÷ naya wall-clock. Humne kam time mein utna hi kaam (2000 insns) kiya, aur woh ratio exactly "kitna tez" ka matlab hai.

Figure dekho: amber blocks (thread A ka kaam) white gaps (thread B ke stalls) mein seedha fit ho jaate hain.

Figure — Simultaneous multithreading (SMT - hyperthreading)

Verify: ek tempting shortcut hai "combined IPC $= 1.8 + 0.6 = 2.4$ , toh time $= 2000/2.4 = 833$ cycles." Yeh galat hai kyunki woh formula assume karta hai ki dono threads poore window mein apni combined rate par run karte hain — lekin thread A actually ruk jaata hai 555.6 cycles par, jiske baad sirf B IPC $0.6$ par run kar raha hai. Toh combined rate sirf pehle 555.6 cycles tak hold karta hai, poori tarah nahi. Sahi answer finishing times se seedha aata hai: B last mein 1666.7 cycles par finish karta hai, aur A already 555.6 par finish ho gaya, toh dono 1666.7 cycles par done hain. Speedup $2500/1666.7 = 1.50 ✓$ . Units: cycles/cycles = dimensionless. ✓

C2 — Do CPU-bound threads (woh collide karte hain)

Forecast: fill karne ke liye almost koi idle units nahi — 1× se barely upar expect karo.

Sequential time. Har ek $1000/2.4 = 416.7$ cycles; total $= 833.3$ cycles. Yeh step kyun? C1 jaisa hi: ek thread completion tak run karo, phir doosra, toh do runtimes add hote hain.
SMT time per thread. $1000/1.4 = 714.3$ cycles. Dono overlap ⇒ wall-clock $= 714.3$ . Yeh step kyun? Do threads symmetric hain (identical IPC), toh woh saath finish karte hain; wall-clock sirf ek thread ka SMT runtime hai.
Speedup. $833.3/714.3 = 1.17$ . Yeh step kyun? Purana wall-clock ÷ naya wall-clock, bilkul C1 ki tarah. Chhota gain kyunki zyaadatar units already busy the.

Verify: SMT ke under combined IPC $= 1.4 + 1.4 = 2.8$ , jo single-thread IPC $2.4$ se zyaada hai — toh total throughput sach mein badha, 1 se upar speedup ke saath consistent. ✓ Utilization $2.4/4 = 60%$ se $2.8/4 = 70%$ tak badhi — sirf 10 points of idle units recoverable the, isliye gain tiny hai. ✓

C3 — Do memory-bound threads (ek bandwidth model)

Forecast: units wide open hain, lekin memory ki pipe narrow hai. Ek gain guess karo, lekin chhota — 2× se kaafi kam.

Sequential time. Har ek $1000/0.5 = 2000$ cycles; total $= 4000$ cycles. Yeh step kyun? Ek thread ek time par; akela thread sirf IPC $0.5$ tak pahunchta hai, toh har ek 2000 cycles leta hai aur woh add hote hain.
SMT ke under har thread ko kya IPC milti hai? Do threads chahte hain $0.5 + 0.5 = 1.0$ combined, lekin pipe sirf $B = 0.9$ deliver karti hai. Toh memory system unhe combined $0.9$ tak throttle kar deta hai, evenly split: har thread ko $0.9/2 = 0.45$ milta hai. Yeh step kyun? Yahi C3 ka poora point hai — ceiling $N$ nahi, $B$ hai. Jab demand ( $1.0$ ) supply ( $0.9$ ) se zyada ho, har thread ko apne fair share tak scale down kiya jaata hai.
SMT time per thread. $1000/0.45 = 2222.2$ cycles. Dono overlap ⇒ wall-clock $= 2222.2$ cycles. Yeh step kyun? Symmetric threads saath finish karte hain, toh wall-clock ek thread ka SMT runtime hai.
Speedup. $4000/2222.2 = 1.80$ . Yeh step kyun? Purana wall-clock ÷ naya wall-clock. Note karo ki gain purely do threads' kaam ek window mein karne se aata hai; throttled bhi ho toh, dono ko ek saath chalana unhe ek ke baad ek chalane se better hai.

Verify: honest ceiling check — combined SMT IPC $= 0.45 + 0.45 = 0.9 = B$ exactly, toh hum exactly bandwidth-saturated hain aur kabhi usse upar nahi. ✓ Naive "har ek $0.5$ par rehta hai" fantasy se compare karo: woh combined $1.0 > B$ demand karega, jo pipe deliver nahi kar sakti, toh $0.45$ tak throttle forced hai. Execution units $0.9/4 = 22.5%$ utilization par baith rahe hain — almost khaali — yeh prove karta hai ki bottleneck memory bandwidth hai, units nahi. ✓ Speedup $1.80$ , forecast ke anusaar 2× se neeche. ✓

C4 — Degenerate case: sirf ek thread exist karta hai

Forecast: bubbles fill karne ke liye koi nahi, SMT ko... kuch nahi karna chahiye.

SMT time. Koi B instructions ready nahi hongi kabhi, toh A bilkul akele ki tarah run karta hai: $1000/2.0 = 500$ cycles. Yeh step kyun? Scheduler ready queue se pick karta hai; agar sirf A ke paas ready ops hain, A har slot leta hai — ek khaali doosre context ki presence kuch nahi badlti.
Single-thread ke against Speedup. $500/500 = 1.00$ . Yeh step kyun? Purana wall-clock ÷ naya wall-clock, aur dono identical hain, toh ratio exactly 1 hai — yeh baseline prove karta hai ki jab overlap karne ke liye kuch nahi, SMT no-op hai.

Verify: per-thread state duplicate (PC, registers) unused pada hai; idle hone par zero performance cost kyunki shared execution units abhi bhi 100% A ke liye available hain. Speedup exactly $1.0 \times$ . ✓ Yeh correct "SMT akele thread ki throughput kabhi hurt nahi karta" baseline hai.

C5 — Limiting / ideal case (2× ceiling)

Forecast: theoretical best — exactly 2× guess karo.

Sequential time. Har ek $1000/2.0 = 500$ ; total $= 1000$ cycles. Yeh step kyun? Ek thread phir doosra; runtimes add hote hain.
SMT time. Koi contention nahi ⇒ har ek IPC $2.0$ par rehta hai ⇒ har ek $500$ cycles, overlapping ⇒ wall-clock $500$ . Yeh step kyun? Kyunki A sirf integer units touch karta hai aur B sirf FP units, woh kabhi compete nahi karte, toh koi slow nahi hota — woh ideal jo koi doosra cell nahi reach karta.
Speedup. $1000/500 = 2.00$ . Yeh step kyun? Purana wall-clock ÷ naya wall-clock. Humne do threads ka kaam ek ke time mein kiya, aur zero slowdown ke saath woh ratio 2-way core ki hard ceiling tak pahunchta hai.

Neeche figure ek resource map hai, timeline nahi. Vertical axis 8 execution units list karta hai (unit 0 neeche, unit 7 upar). Amber blocks woh 4 integer units hain jo thread A ne claim ki hain; cyan blocks woh 4 FP units hain jo thread B ne claim ki hain. Key visual takeaway: do colours koi row share nahi karte, toh combined demand ( $4$ units of A $+$ $4$ of B $= 8$ ) $N = 8$ ke andar fit ho jaati hai bina kisi overlap ke — woh geometric non-collision exactly kyun hai 2× ceiling yahan reachable hai.

Verify: combined IPC $= 4.0$ , aur $4.0 \leq N = 8$ , toh koi execution-unit conflict nahi — ceiling reachable hai sirf isliye kyunki units abundant hain aur workloads disjoint hain. $1000/500 = 2.0 ✓$ . Yeh 2-way SMT ki absolute upper bound hai.

C6 — Amdahl sweep (formula check, do points)

$f = 0.6$ par: denominator $= (1 - 0.6) + 0.6/2 = 0.4 + 0.3 = 0.7$ . Speedup $= 1/0.7 = 1.4286$ . Yeh step kyun? 60% cycles half-fillable bubbles offer karte hain, toh hum kaam ke $f$ ko $f /2$ effective time se replace karte hain.
$f = 0.9$ par: denominator $= 0.1 + 0.45 = 0.55$ . Speedup $= 1/0.55 = 1.8182$ . Yeh step kyun? Almost sab stall time hai ⇒ $f \to 1$ limit $1/0.5 = 2.0$ ki taraf approach kar raha hai.

Neeche plot mein $f$ (memory-bound fraction, 0 se 1) horizontal axis par hai aur speedup vertical axis par. Cyan curve $Speedup (f)$ hai; do amber dots hamare worked points $f = 0.6$ aur $f = 0.9$ mark karte hain; dashed white line $2.0 \times$ ceiling hai. Visual takeaway: curve 1.0 (koi stalls nahi, koi help nahi) se 2.0 (sab stalls, sibling aadha fill karta hai) ki taraf uthta hai aur ceiling ke paas flatten ho jaata hai — diminishing returns dikhata hai jab workloads zyada memory-bound hote jaate hain.

Verify: limits — $f = 0$ par (sab CPU-bound) speedup $= 1/ (1) = 1.0$ ✓; $f = 1$ par speedup $= 1/0.5 = 2.0$ ✓; unke beech monotonic increasing ✓. Hamare do points $1.4286$ aur $1.8182$ us curve par hain. ✓

C7 — Real-world word problem (ek web server)

Forecast: SMT throughput ko C6 factor $1.4286$ se multiply karta hai.

Cycles per request (single thread). $6000/1.5 = 4000$ cycles. Yeh step kyun? Instructions ÷ IPC = cycles; yeh "work per request" ko "time per request" mein convert karta hai.
Requests/sec, no SMT. $3 \times 1 0^{9} /4000 = 750, 000$ req/s. Yeh step kyun? Cycles-per-second ÷ cycles-per-request = requests-per-second; clock rate set karta hai ki ek second mein kitne request-sized time-slices fit hote hain.
SMT speedup apply karo (C6 se, $f = 0.6$ ): $750, 000 \times 1.4286 = 1, 071, 429$ req/s. Yeh step kyun? Kyunki is workload ka stall fraction exactly wahi $f = 0.6$ hai jo humne C6 mein sweep kiya, uska aggregate throughput usi Amdahl factor se badh ta hai — yahan abstract curve ek concrete "requests per second" ban jaati hai, CPU-vs-memory mix idea ko directly reuse karte hue.

Verify: effective IPC ke through cross-check. Speedup $1.4286$ matlab aggregate IPC $= 1.5 \times 1.4286 = 2.143$ ; cycles per request effectively $6000/2.143 = 2800$ ; $3 \times 1 0^{9} /2800 = 1, 071, 429$ req/s ✓ — step 3 se match karta hai. Units: (cycles/s)/(cycles/req) = req/s ✓. Wrap-up: server ab lagbhag 43% zyada requests per second serve karta hai ~5% die-area cost par, aur single web request khud nahi teza hoti — throughput up, per-request latency flat-to-slightly-up, exactly woh SMT trade jo topic note ne promise ki thi.

C8 — Exam twist: jab SMT hurt karta hai

Forecast: agar per-thread IPC kaafi hard crash kar jaaye, toh aggregate throughput unhe ek ek karke run karne se neeche gir sakti hai. Guess karo: speedup < 1.0?

Sequential time. Har ek $1000/2.0 = 500$ ; total $= 1000$ cycles. Yeh step kyun? Ek thread phir doosra; runtimes add hote hain — yeh woh baseline hai jise SMT ko beat karna chahiye.
SMT time. Har ek $1000/0.8 = 1250$ ; overlap ⇒ wall-clock $1250$ . Yeh step kyun? Symmetric threads saath finish karte hain, toh wall-clock ek thread ka SMT runtime hai; catch yeh hai ki har thread ka IPC 2.0 se 0.8 tak collapse ho gaya kyunki woh ek doosre ki cache lines evict kar rahe hain.
Speedup. $1000/1250 = 0.80$ . Yeh step kyun? Purana wall-clock ÷ naya wall-clock — aur yahan ratio 1 se neeche hai, matlab SMT ne total throughput worse kar diya, woh counter-intuitive result jo is cell ko dikhane ke liye exist karta hai.

Verify: SMT ke under combined IPC $= 0.8 + 0.8 = 1.6 <$ single-thread $2.0$ — toh total throughput actually gir gayi. Speedup $0.8 < 1$ ✓. Asli wajah: shared cache capacity per thread half ho gayi + predictor interference. Isliye SMT ek BIOS/OS toggle hai: cache-thrashing workloads ke liye tum use off kar dete ho.

Recall Self-test: cell ko outcome se match karo

Kaun sa cell exactly 1.0× deta hai aur kyun? ::: C4 — akela thread idle sibling ke saath; bubbles fill karne ke liye kuch nahi, koi contention nahi, toh SMT no-op hai. Kaun sa cell exactly 2.0× deta hai aur kyun? ::: C5 — wide core par disjoint resource use; combined IPC $N$ ke under rehti hai, toh zero contention theoretical ceiling hit karta hai. Kaun sa cell < 1.0× deta hai aur kyun? ::: C8 — cache thrashing per-thread IPC itna gira deta hai ki aggregate throughput sequential se neeche chali jaati hai. $f = 0.6$ par, Amdahl kya speedup predict karta hai? ::: $1/ ((1 - 0.6) + 0.6/2) = 1/0.7 = 1.43 \times$ .

Connections

Superscalar Processors — bahut se execution units wahi hain jo SMT busy rakhta hai.
Thread-Level Parallelism (TLP) — SMT ek core ke andar realize hota TLP hai.
Pipeline Hazards — stall bubbles exactly woh slots hain jo SMT fill karta hai.
Branch Prediction — shared predictor interference ek C8-style penalty hai.
Cache Coherence / Memory Hierarchy — shared caches dono wins (C1) aur losses (C8) drive karte hain.