Multicore vs manycore designs

6.1.4 · Hardware › Parallelism & Multicore

Core Concept

Modern parallel processors do fundamental architectural philosophies mein aate hain: multicore (thode powerful cores) aur manycore (bahut saare simpler cores). Yeh design choice ek fundamental tradeoff represent karta hai per-thread performance aur throughput capacity ke beech.

Key insight yeh hai: Dono ek saath nahi ho sakte. Transistor budget fixed hota hai. Jo paisa complex out-of-order execution logic, bade caches, aur branch prediction pe lagta hai, woh paisa zyada cores add karne pe nahi lagta.

Detailed Comparison

Multicore Architecture

Design Philosophy: Single-thread performance maximize karo. Har core complex, irregular code ko unpredictable branches aur memory access patterns ke saath efficiently execute kar sakta hai.

Yeh architecture kyun?

1. Legacy code optimization: Zyaatar software single-threaded execution ke liye likha jaata hai
2. Latency-critical tasks: Operating systems, databases, compilers fast sequential execution se fayda uthate hain
3. Irregular parallelism: Unpredictable control flow wale tasks (web servers, transaction processing)

$\text{Speedup}=\frac{1}{(1-P)+\frac{P}{N}}$

Jahaan:

• $P$ = code ka woh fraction jo parallelize ho sakta hai
• $N$ = cores ki sankhya
• $(1-P)$ = serial fraction (asli villain)

First principles se derivation: Total execution time ke do components hain: $T_{\text{total}}=T_{\text{serial}}+T_{\text{parallel}}$

$N$ cores ke saath, sirf parallel portion speed up hota hai: $T_{\text{multicore}}=T_{\text{serial}}+\frac{T_{\text{parallel}}}{N}$

Original time $T_{\text{orig}}=T_{\text{serial}}+T_{\text{parallel}}$ se normalize karke: $\text{Speedup}=\frac{T_{\text{orig}}}{T_{\text{multicore}}}=\frac{T_{\text{serial}}+T_{\text{parallel}}}{T_{\text{serial}}+\frac{T_{\text{parallel}}}{N}}$

Numerator aur denominator dono ko $T_{\text{orig}}$ se divide karke: $\text{Speedup}=\frac{1}{\frac{T_{\text{serial}}}{T_{\text{orig}}}+\frac{T_{\text{parallel}}}{N\cdot T_{\text{orig}}}}=\frac{1}{(1-P)+\frac{P}{N}}$

Critical insight: Agar sirf 5% bhi serial hai ($P=0.95$), toh max speedup $\frac{1}{0.05}=20\times$ hai, chahe kitne bhi cores kyun na hon. Isliye multicore har core ko fast rakhne pe focus karta hai.

Manycore Architecture

Design Philosophy: Massive parallelism ke through aggregate throughput maximize karo. Instruction-level tricks ki bajaye thread-level parallelism se latency hide karo.

Yeh architecture kyun?

1. Data parallelism: Massive datasets par same operation (SIMT - Single Instruction Multiple Threads)
2. Throughput-oriented: Scientific computing, graphics, machine learning
3. Latency tolerance: Memory stalls ke dauran doosre thread par switch kar sako (100+ cycles)

$\text{Throughput}=N_{\text{cores}}\times f_{\text{clock}}\times \text{IPC}\times \text{Utilization}$

Derivation:

• Har core $\text{IPC}$ (instructions per cycle) × $f_{\text{clock}}$ (cycles/sec) instructions/second complete karta hai
• $N_{\text{cores}}$ ke saath, raw capacity $N\times f\times \text{IPC}$ hai
• Utilization factor (0-1) thread divergence, memory bottlenecks account karta hai

Key design choice: $N$ dramatically badhane ke liye IPC aur $f$ per core kam karo.

Example calculation:

• Multicore: 8 cores × 4 GHz × 4 IPC × 0.9 util = 115.2 GIPS
• Manycore: 512 cores × 1.5 GHz × 1 IPC × 0.7 util = 537.6 GIPS (4.7× zyada throughput)

Lekin single thread par, multicore $4\times 4=16\times$ faster execute karta hai per clock!

Example 1: Intel Core i9 (Multicore)

• Architecture: 8-16 high-performance cores
• Per-core features:
  • 512KB L2 cache per core
  • 6-wide superscalar execution
  • 32MB shared L3 cache
  • Hyper-Threading (2 threads/core)
• Use case: Gaming, content creation, general computing
• Yeh kyun kaam karta hai: Games mein complex AI, physics, rendering pipelines hoti hain jo fast sequential execution se fayda uthati hain. 8 threads ke saath bhi, single-thread performance matter karti hai (ek thread aksar bottleneck banta hai).

Step-by-step performance analysis:

1. Game physics engine mein 30% parallel, 70% serial code hai
2. 8 cores ke saath: Speedup = $\frac{1}{0.7+\frac{0.3}{8}}=\frac{1}{0.7375}=1.36\times$
3. Lekin har core 4.5 GHz par high IPC ke saath chalta hai → absolute performance high hai
4. Yeh step kyun? Modest parallel speedup bhi acceptable hai jab base performance excellent ho.

Example 2: NVIDIA A100 GPU (Manycore)

• Architecture: 6912 CUDA cores (108 SMs × 64 cores each)
• Per-core features:
  • Bahut simple in-order pipeline
  • Tiny caches, massive thread count
  • Hardware thread scheduler (32-wide warps)
• Use case: Deep learning training, scientific simulation
• Yeh kyun kaam karta hai: Matrix multiplication 100% parallel hai. Neural network training mein millions of weights independently process hote hain.

Step-by-step performance analysis:

1. Matrix multiplication: $C=A\times B$ jahaan $A,B$ hain $1024\times 1024$
2. $1024^2=1,048,576$ output elements mein se har ek independently compute hota hai
3. Har element ko ek thread assign karo: 100% parallel ($P=1.0$)
4. Speedup = $\frac{1}{0+\frac{1}{6912}}\approx 6912\times$ (theoretical)
5. Yeh step kyun? Perfect parallelism ke saath, zyada cores = linear speedup.
6. Actual: ~4000× memory bandwidth limits ki wajah se, lekin phir bhi multicore se kaafi aage.

Example 3: Database Query (Multicore Wins)

• Task: 5 tables ke saath complex JOIN, WHERE clauses, aggregations
• Multicore kyun?
  1. Query planning: sequential logic, unpredictable branches
  2. Index traversals: pointer chasing (manycore ke liye bahut bura)
  3. Lock management: sequential critical sections
  4. Parallel scan ke saath bhi, complexity fast cores ko favor karti hai

The Resource Tradeoff

Ek chip ke paas fixed resources hoti hain:

$A_{\text{total}}=N_{\text{cores}}\times (A_{\text{core}}+A_{\text{cache}}+A_{\text{control}})+A_{\text{interconnect}}$

Jahaan:

• $A_{\text{total}}$ = total die area (process node aur cost se fixed)
• $A_{\text{core}}$ = har core ke execution units ka area
• $A_{\text{cache}}$ = cache area per core
• $A_{\text{control}}$ = out-of-order logic, branch prediction, etc.
• $A_{\text{interconnect}}$ = core communication ke liye network-on-chip

Tradeoff ki derivation: Multicore ke liye: $A_{\text{control}}$ aur $A_{\text{cache}}$ BADE hain (core area ka 50-60%) Manycore ke liye: $A_{\text{control}}\approx 0$, $A_{\text{cache}}$ minimal hai (core area ka 10-15%)

Agar ek complex core $A_{\text{complex}}=10{\text{ mm}}^2$ area leta hai, aur simple core $A_{\text{simple}}=0.2{\text{ mm}}^2$ leta hai:

$N_{\text{simple}}=\frac{A_{\text{total}}}{A_{\text{simple}}}=50\times N_{\text{complex}}$

Yeh kyun matter karta hai: Tum 1 complex core ke badle ~50 simple cores lete ho. Yeh TABHI kaam karta hai jab tumhara workload saare 50 ko efficiently use kar sake.

Memory Access Patterns

Multicore strategy: Complex caching aur out-of-order execution se latency hide karo

• Bade caches working set ko capture karte hain (temporal locality)
• Out-of-order execution memory stalls ke dauran independent instructions dhundhta hai
• Hardware prefetchers access patterns predict karte hain
• Cost: Bahut zyada silicon area, power

Manycore strategy: Massive thread-level parallelism se latency hide karo

• Jab thread 1 memory ka wait kare, turant thread 2 par switch karo
• Core per 100 threads ke saath, hamesha kaam milta hai
• SIMD/SIMT chahiye (saare threads same instruction execute karein)
• Cost: Hazaron parallel threads chahiye; irregular code mein help nahi karta

Scenario: DRAM se load karne mein 300 cycles lagte hain

Multicore approach (Intel core):

1. Load instruction issue karo
2. Out-of-order engine window mein 100+ independent instructions dhundhta hai
3. Wait karte waqt woh execute karo
4. Agar independent work khatam ho jaaye, core stall karta hai (bura!)
5. Yeh step kyun? Latency hide karne ke liye single thread ke andar ILP exploit karna.

Manycore approach (GPU):

1. Thread 1 load instruction issue karta hai → context switch (1 cycle)
2. Thread 2 execute hota hai → context switch
3. Thread 3 execute hota hai → ...
4. Thread 32 execute hota hai → wapas thread 1 par, data likely aa chuka hai
5. Yeh step kyun? Kaafi threads ke saath, kabhi stall nahi; koi na koi hamesha ready hota hai.

Common Misconceptions

Kyun galat hai: Speedup serial fraction (Amdahl's Law) SE AUR communication overhead se limited hota hai.

Fix yeh hai: $\text{Real Speedup}=\frac{1}{(1-P)+\frac{P}{N}+C(N)}$

jahaan $C(N)$ communication/synchronization overhead hai jo $N$ ke saath BADHTA hai.

1000 cores ke saath, agar synchronization har core ka 1% time leta hai: $C(N)=0.01\times N=10$ → negative speedup!

Example: 1000 cores ke saath 1M numbers sort karna:

• Data partition karo: $O(N)$ kaam, negligible
• Locally sort karo: $O(\frac{N}{P}\log \frac{N}{P})$ - perfect parallelism
• Results merge karo: $O(N\log P)$ - serial bottleneck
• $P=1000$ ke liye, merge time dominate karta hai jab tak $N$ enormous na ho

Kyun galat hai: Sirf data-parallel workloads ke liye sach hai. GPUs inn chezon mein bahut bure hain:

• Branching code (if-else "warp divergence" cause karta hai - cycles waste hote hain)
• Pointer chasing (linked lists, trees)
• Chhote datasets (kernels launch karne ka overhead)
• Irregular memory access

Fix yeh hai: Architecture ko workload pattern se match karo.

Example - Tree traversal:

// GPU par bahut bura
struct Node { int value; Node* left; Node* right; };
int search(Node* root, int target) {
    if (!root) return -1;
    if (root->value == target) return 1;
    return search(root->left, target) || search(root->right, target);
}

GPU kyun fail karta hai:

1. Har thread alag path follow karta hai (warp divergence)
2. Pointer chasing: 100% cache misses, latency hide nahi ho sakti
3. Koi data parallelism nahi: har search dependent hai

Branch prediction + large cache wala CPU 10×+ se jeet jaata hai.

Kyun galat hai: Serial portion ABHI BHI clock speed par chalta hai. Amdahl's Law ke according, serial portion ultimate speedup ceiling determine karta hai.

Fix yeh hai: Balance matter karta hai. Bahut slow cores even parallel code ko hurt karte hain.

Example: Video encoding (90% parallel)

• Fast cores (4 GHz): Serial 10s leta hai, parallel 90s → 8 cores ke saath: $10+\frac{90}{8}=21.25s$
• Slow cores (1 GHz): Serial 40s leta hai, parallel 360s → 32 cores ke saath: $40+\frac{360}{32}=51.25s$

4× zyada cores ke bawajood, slow architecture serial bottleneck ki wajah se 2.4× SLOWER hai.

Design Decision Framework

Multicore use karo jab: $\frac{T_{\text{serial}}}{T_{\text{total}}}>0.05\text{ OR branching/irregular memory}$

Manycore use karo jab: $P>0.95\text{ AND }{N}_{\text{available\_threads}}\gg N_{\text{cores}}$

Formal criteria:

1. Parallelism availability: Kya tum manycore cores se $10\times -100\times$ zyada threads bana sakte ho?
2. SIMD/SIMT viability: Kya threads same instruction stream execute karte hain?
3. Memory access pattern: Regular (strided, coalesced) ya irregular (random)?
4. Latency tolerance: Kya tum 10-100× zyada single-thread latency afford kar sakte ho?

MANYcore: Maximize Agregate, No branches, Yield for latency

Ya socho: BIG·few vs little·many

Connections

• Amdahl's Law - parallel speedup ko limit karne wale serial bottleneck ko quantify karta hai
• Thread-Level Parallelism - multi vs manycore se alag tarike se exploit hoti hai
• Cache Coherence - multicore shared memory ke liye critical, manycore ke liye kam relevant
• SIMD vs MIMD - manycore SIMT use karta hai (GPU model), multicore MIMD hai
• Memory Hierarchy - multicore caches emphasize karta hai, manycore bandwidth emphasize karta hai
• Power Consumption - manycore parallel workloads ke liye performance-per-watt mein jeet jaata hai
• Heterogeneous Computing - modern systems dono combine karte hain (CPU+GPU)

Multicore aise hai jaise 8 bahut smart teachers hon. Har teacher koi bhi test super fast grade kar sakta hai kyunki woh expert hain. Woh weird questions handle kar sakte hain, kaam check kar sakte hain, partial credit de sakte hain - sab complex kaam. Agar sirf 8 tests hain, bahut achha! Lekin agar 1000 tests hain jo saare same hain (sirf multiple choice), toh woh 8 teachers ka talent waste ho raha hai.

Manycore aise hai jaise 500 fifth-graders ke paas answer keys hon. Har baccha SIRF check kar sakta hai ki answer key se match karta hai ya nahi. Woh individually smart nahi hain, lekin UN MEIN SE BAHUT SAARE HAIN. Agar test saara multiple choice hai (simple, repetitive), woh 1000 tests ko 8 teachers se kaafi jaldi khatam kar denge. Lekin agar ek bhi question mein explanation chahiye, saare 500 bacche stuck ho jaate hain.

Computer chip teri school budget jaisi hai - tum ya toh 8 expert teachers hire kar sakte ho YA 500 simple checkers, dono nahi!

#flashcards/hardware

Multicore aur manycore processors mein key difference kya hai? :: Multicore mein thode (2-16) complex, high-performance cores hote hain jo single-thread speed ke liye optimized hain; manycore mein bahut saare (100s-1000s) simple cores hote hain jo aggregate throughput ke liye optimized hain.

Concept Map