Simultaneous multithreading (SMT - hyperthreading)

5.3.14 · Hardware › Advanced Microarchitecture

Overview

Simultaneous Multithreading (SMT), commercially Intel ke dwara hyperthreading ke naam se brand kiya gaya, ek microarchitectural technique hai jo ek single physical processor core ko same clock cycle mein multiple hardware threads se instructions execute karne deti hai. Yeh un execution units ki utilization maximize karta hai jo otherwise pipeline stalls, cache misses, ya instruction dependencies ki wajah se idle baith jaate hain.

Ise carpooling ki tarah socho: ek car (core) multiple passengers (threads) ko carry karti hai khaali seats utilize karne ke liye. Car faster nahi jaati, lekin zyada log ek trip mein apni destination tak pahunch jaate hain.

Problem: Underutilized Execution Resources

First Principles se Derivation

Ek typical out-of-order superscalar processor mein hota hai:

Frontend: instruction fetch, decode (shared)
Execution units: 4-6 ALUs, 2-3 FPUs, 2 load/store ports (shared)
Backend: register renaming, reorder buffer (ROB), commit logic (shared)

Ek single thread ke liye, ideal instruction-level parallelism (ILP) extraction ka matlab hai:

IPC_{ideal} = min (Dispatch Width, Available ILP)

Lekin real programs mein hote hain:

Data dependencies: instruction $I_{2}$ , $I_{1}$ ka result aane ka wait karta hai
Cache misses: DRAM ke liye wait karte hue 50-200 cycle stalls
Branch mispredictions: pipeline flushes

Cache miss ke dauran, saare execution units idle hote hain kyunki thread ke paas koi ready instructions nahi hoti. Utilization:

Utilization_{single} = \frac{Active cycles}{Total cycles} \approx 20 - 40%

Key insight: Hardware exist karta hai lekin unused baith jaata hai. Kya hum un bubbles ko doosre thread ke kaam se fill kar sakte hain?

SMT Kaise Kaam Karta Hai

Architecture Changes (Kya Duplicate vs Shared Hota Hai)

Har thread ke liye Duplicate (sasta):

Program Counter (PC)
Architectural register file (32-64 registers × 2 threads = 128 total)
Thread state (privilege level, exception vectors)

Cost: 2-way SMT ke liye ~5% die area increase.

Execution units (ALUs, FPUs, load/store ports)
Caches (L1, L2, L3)
Branch predictor
TLB (translation lookaside buffer)

Scheduling Mechanism

Har cycle mein, instruction scheduler (out-of-order engine ka part) saare threads ke instructions ki ek ready queue maintain karta hai. Selection policy:

Select = ar g thread max (Priority \times Ready Instructions)

Round-robin with stall priority: Agar thread A stall kare (cache miss), toh thread B execution units par monopolize karta hai jab tak A unblock na ho jaaye.

Derivation: Throughput Gain

Single-Thread Baseline

Maano:

Core ke paas $N = 4$ execution units hain
Thread A ka IPC = 1.5 hai (stalls ki wajah se average mein 1.5 units/cycle use karta hai)
Utilization = $\frac{1.5}{4} = 37.5%$

1000 instructions ke liye time:

T_{single} = \frac{1000}{1.5} = 667 cycles

Two-Way SMT

Thread B simultaneously run karne ke saath:

Thread A: IPC = 1.2 (resource contention ki wajah se thodi slowdown)
Thread B: IPC = 1.2
Combined throughput: $1.2 + 1.2 = 2.4$ IPC
Utilization: $\frac{2.4}{4} = 60%$

Har thread ke liye 1000 instructions ka time:

T_{SMT} = \frac{1000}{1.2} = 833 cycles per thread

Lekin: Dono threads usi 833 cycles mein finish hote hain, isliye total kaam = 833 cycles mein 2000 instructions.

Throughput gain:

Speedup = \frac{2000/833}{1000/667} = \frac{2.4}{1.5} = 1.6 \times

2× kyun nahi? Threads compete karte hain:

Cache capacity (zyada misses)
TLB entries (zyada page faults)
Reorder buffer slots (out-of-order window ko limit karta hai)

Worked Examples

SMT ke bina:

Thread A: IPC = 2.0, time = 500 cycles
Thread B: IPC = 0.5, time = 2000 cycles
Total time: 500 + 2000 = 2500 cycles (sequential)

SMT ke saath:

Jab B stall karta hai, A execution units use karta hai
Thread A: IPC = 1.8 (thodi slowdown), time = 556 cycles
Thread B: IPC = 0.6 (thodi speedup), time = 1667 cycles
Dono finish hote hain: max(556, 1667) = 1667 cycles

Yeh step kyun? Hum B ke stalls ko A ke computation ke saath overlap kar rahe hain. Long pole B ki memory latency hai, lekin A un cycles ke 70% ko "fill in" karta hai.

Result: 2500 → 1667 cycles = 1.5× speedup.

SMT ke bina:

Har ek: IPC = 2.5, time = 400 cycles
Total: 800 cycles sequential

SMT ke saath:

Resource contention har ek ko IPC = 1.4 tak reduce karta hai
Har thread ka time: 714 cycles
Dono finish hote hain: 714 cycles

Yeh step kyun? Limited execution units 50/50 split hote hain. Exploit karne ke liye koi idle resources nahi hain.

Result: 800 → 714 cycles = 1.12× speedup (minimal gain).

Lesson: SMT tab sabse zyada help karta hai jab threads ki complementary resource needs hoti hain.

Performance Modeling

Speedup = \frac{1}{( 1 - f ) + \frac{f}{2}}

Derivation:

CPU-bound fraction: $(1 - f)$ → koi benefit nahi (saturated units)
Memory-bound fraction: $f$ → second thread 50% bubbles fill karta hai (conservative)

Yeh formula kyun? Memory-bound cycles mein, second thread idle units use kar sakta hai. CPU-bound cycles mein, threads compete karte hain. Yeh ek simplified model hai; real speedup cache behavior par depend karta hai.

Example: Agar $f = 0.6$ (60% memory-bound):

Speedup = \frac{1}{0.4 + 0.3} = 1.43 \times

Typical range: General workloads ke liye 1.2× to 1.4×.

Common Mistakes

Steel-man: Agar execution units stalls ke dauran 100% idle hain, aur second thread ki zero contention hai, toh yeh sach hota. Practice mein:

Shared caches → zyada misses → dono ke liye lambe stalls
Limited reorder buffer → out-of-order window restrict karta hai
Branch predictor thrashing → zyada mispredictions

Reality: 20-40% throughput gain typical hai. Single-thread latency often badhti hai (har task mein zyada time, lekin zyada tasks per second).

Steel-man: Yeh ek coarse-grained time-sliced system ke liye sach hota. Lekin SMT fine-grained hai: instructions har cycle mein readiness ke basis par select hoti hain. Memory par stalled thread execution units ka 0% use kar sakta hai jabki doosra 100% use karta hai.

Fix: "Static partitioning" nahi, "opportunistic sharing" socho.

Steel-man: Infinite execution resources ke saath, yeh sach hota. Lekin:

Zyada threads → zyada cache thrashing
Zyada threads → bada architectural state (PC, registers) overhead
2-4 threads per core ke baad diminishing returns

Reality: Zyaatar commercial CPUs 2-way (Intel) ya 4-way (IBM POWER) SMT par ruk jaate hain.

Connections Out-of-Order Execution: SMT, OoO par build karta hai threads ke across instruction pool expand karke

Superscalar Processors: Multiple execution units SMT ki parallelism enable karte hain
Cache Coherence: Shared L1/L2 caches ko coherence protocols chahiye jab threads shared data access karte hain
Thread-Level Parallelism (TLP): SMT microarchitecture level par TLP hai (OS-level multithreading ke mukable)
Pipeline Hazards: SMT structural hazards ko idle units borrow karke mitigate karta hai
Branch Prediction: Shared predictor inter-thread interference se suffer kar sakta hai
Memory Hierarchy: Cache miss latency SMT ke liye primary opportunity hai

Recall Ek 12-Saal ke Bachche ko Explain Karo

Socho tum homework kar rahe ho aur tumhare paas ek pencil, eraser, calculator, aur ruler hai. Kabhi kabhi tum pencil se likhte ho, lekin calculator wahan bas baith jaata hai. Tumhare dost ko bhi homework karna hai. Desk par baari baari use karne ki jagah, tum DONO ek saath desk par baith jaate ho! Jab tum likh rahe hote ho (pencil use karke), tumhara dost calculator use karta hai. Jab tum soch ke atak jaate ho, tumhara dost pencil use karta hai. Tum tools share karte ho, lekin tum dono apna apna homework karte ho. Desk bada nahi hota, lekin zyada homework hota hai kyunki tools yahan wahan idle nahi baith rahe. Yahi hyperthreading hai—ek "desk" (CPU core) do "students" (programs) ki madad karta hai unhe tools (execution units) smartly share karne deke.

Flashcards

Simultaneous Multithreading (SMT) kya hai? :: Ek microarchitectural technique jisme ek single physical core multiple hardware thread contexts (PC, registers) maintain karta hai aur same clock cycle mein multiple threads se instructions execute kar sakta hai, un execution units ko utilize karte hue jo otherwise stalls ke dauran idle rehte.

SMT mein kon se hardware resources har thread ke liye DUPLICATE hote hain?

Program Counter (PC), architectural register file, thread state (privilege level, exception vectors). Cost: 2-way SMT ke liye ~5% die area.

SMT mein kon se hardware resources threads ke across SHARED hote hain?

Execution units (ALUs, FPUs), caches (L1/L2/L3), branch predictor, TLB, reorder buffer. Yeh duplicate karne ke liye bahut mehange hain.

SMT 2 threads ke saath 2× performance kyun nahi deta?

Kyunki threads shared resources ke liye compete karte hain: cache capacity (zyada misses), TLB entries (zyada page faults), reorder buffer slots (OoO window limit karta hai), aur branch predictor accuracy (interference).

SMT kab SABSE ZYADA benefit deta hai?

Jab threads ki complementary resource usage ho: ek thread memory-bound ho (bahut saare stalls) jabki doosra CPU-bound ho (idle execution units use kar sake). Typical gain: 1.2-1.4× throughput.

2-way SMT se typical throughput speedup kitna hota hai?

General workloads ke liye 1.2× to 1.4×. Ideal complementary threads (memory-bound + CPU-bound) ke saath 1.6× tak pahunch sakta hai. Jab dono threads CPU-bound hon tab minimal (<1.1×).

SMT single-thread latency ko kaise affect karta hai?

Single-thread latency often BADHTI HAI kyunki thread ko zyada cache misses aur resource contention face karna padta hai. SMT THROUGHPUT (work per second) improve karta hai na ki LATENCY (time per task).

SMT mein instruction selection policy kya hai?

Har cycle mein, scheduler kisi bhi ready thread se instructions pick karta hai execution slots fill karne ke liye. Typically round-robin with stall priority: agar ek thread stall kare (cache miss), doosra units monopolize karta hai jab tak woh unblock na ho.

Amdahl's Law use karke, agar 60% cycles memory-bound hain, toh SMT speedup kitna hoga?

Speedup = 1 / ((1 - 0.6) + 0.6/2) = 1 / 0.7 = 1.43×. Formula hai 1/((1-f) + f/2) jahan f memory-bound fraction hai.

Zyaatar CPUs 8-way ki jagah 2-way ya 4-way SMT par kyun ruk jaate hain?

Diminishing returns: zyada threads excessive cache thrashing, TLB pressure, aur branch predictor interference cause karte hain. Architectural state overhead badhta hai. 4 threads se aage performance often degrade ho jaati hai.