Warps and warp scheduling: Visual walkthrough — Hinglish deep dive

6.2.5 · D2 · Hardware › GPU Architecture › Warps and warp scheduling

Yeh page parent result ko — Warps and warp scheduling ki latency-hiding formula ko — bilkul scratch se rebuild karta hai. Hum har symbol ko ek picture ke saath earn karenge pehle use karne se. Akhir tak aap samjhoge, apni ragon mein, kyun ek GPU dozens of warps ko paas rakhta hai jab ek waqt mein sirf ek hi run karta hai.

Woh ek equation jis ki taraf hum badh rahe hain:

$N_{warps needed} = ⌈ \frac{L}{W} ⌉$

Abhi yeh sirf ghunghraale symbols hain. Aao ise obvious banate hain.

Step 1 — Warp actually kya hota hai (the marching band)

KYA HAI: Ek warp 32 threads ka ek bundle hota hai. Ek thread ek worker hai jo aapke program ki ek copy karta hai. Ek instruction ek baar fetch hoti hai aur saare 32 ek saath execute karte hain.

YEH GROUPING KYUN: Ek instruction ko fetch aur decode karna expensive hai. Agar aap yeh ek baar karo aur 32 workers maanein, toh woh cost 32 taraf split ho jaati hai. Yahi hai GPU throughput ka poora trick (iske saath compare karo SIMT-vs-SIMD).

PICTURE: Neeche 32 boxes ki row ek warp hai. Red box thread 0 hai; har box ek hi tick par move karta hai.

Step 2 — "Ek instruction" kitna cost karta hai vs. memory read kitna cost karta hai

KYA HAI: Zyaadatar instructions (add, multiply, compare) roughly ek tick mein khatam ho jaati hain — ek cycle. Ek cycle GPU ki dhadkan hai, uski clock ki ek tick. Lekin ek memory read — door DRAM se koi value maangna — saikdon ticks lena wapas aane mein.

YEH KYUN MATTER KARTA HAI: Agar ek warp memory maange aur phir sirf wait kare, toh woh saikdon cycles ke liye frozen baith jaata hai. Un cycles ke dauran uski execution unit kuch nahi karti. Woh idle time dushman hai. Is page par jo kuch bhi hai woh usi ko fill karne ke liye hai.

PICTURE: Neeche, ek sasti instruction ek chhoti si bar hai (1 cycle). Memory read ek lambi red bar hai ( $L$ cycles). Dekho red kitni badi hai.

Step 3 — Akela ek warp apna zyaadatar waqt barbaad karta hai

KYA HAI: Ek single warp ko ek loop mein follow karo: woh thoda kaam karta hai, phir memory read karta hai, phir wait karta hai, phir repeat.

YEH PEHLE KYUN DIKHAYEIN: Problem ko feel karne ke liye ilaaj se pehle. Hume exactly measure karna hai kitna waqt barbaad ho raha hai taaki cure ka ek number ho.

$W$ ko wo useful instructions ki ginti maano jo ek warp memory reads ke beech run karta hai. (Ek instruction ≈ ek cycle, toh $W$ bhi roughly busy-cycles per read hai.)

PICTURE: Warp 0 ki zindagi. Chhota kala = $W$ cycles real kaam ka. Lamba red = $L$ cycles poore wait ka. Red ne kale ko dhak liya hai.

Step 4 — Idea: doosra warp gap mein daalo

KYA HAI: Jab warp 0 apni memory read fire karta hai aur stall karta hai, warp scheduler (SM ke andar ka hardware) turant warp 1 par switch karta hai aur uska kaam run karta hai.

SWITCH COST KYUN NAHI LAGTI: Har warp apne registers aur apna program counter (ek program counter sirf ek pointer hai "main kin instruction par hoon") rakhta hai. Switching = ek pointer update. Koi saving nahi, koi restoring nahi. Yeh zero-overhead switching hai. (CPU se compare karo, jise state save/restore karna padta hai — register partition cost ke liye dekho Register-Pressure.)

PICTURE: Warp 0 ki red waiting-bar ke andar ab warp 1 ki kali work-bar tuck hai. Idle hole bhar raha hai.

Step 5 — Hole ko completely fill karne ke liye kitne warps chahiye?

KYA HAI: Har warp jo hum laate hain woh $W$ cycles ka kaam $L$ -cycle hole mein deta hai. Hum warps add karte rehte hain jab tak unka combined kaam poore hole ko cover na kar le.

DIVISION KYUN: "Size $W$ ke kitne chunks ek size $L$ ke gap mein fit hote hain?" yahi literally woh sawaal hai jo $\frac{L}{W}$ answer karta hai. Division tool hai kyunki hum equal-sized work blocks ( $W$ ) ko ek fixed span ( $L$ ) mein pack kar rahe hain.

PICTURE: Warp 0 ka red hole, ab warps 1, 2, 3, … ke $W$ -sized kale blocks se tile kiya gaya. Gino kitne blocks lagte hain right edge tak pahunchne mein.

Step 6 — Ceiling kyun, plain division nahi

KYA HAI: $\frac{L}{W}$ fractional aa sakta hai — jaise $\frac{400}{30} = 13.33$ . Aap ek warp ka teesra hissa schedule nahi kar sakte.

ROUND UP (ceiling) KYUN: Agar aap down round karo 13 warps par, unka kaam $13 \times 30 = 390$ cycles hai — woh ek 10-cycle gap chhodta hai jahan machine phir idle ho. Koi bhi bacha hua hole matlab waste throughput, toh hume up round karna hai 14 tak. Ceiling $⌈ x ⌉$ ka matlab hai "sabse chhota poora number $\geq x$ " — yeh precisely isi liye exist karta hai ki hum kabhi gap na chhodein.

PICTURE: 13 blocks kam padte hain (ek patla red sliver bachta hai). 14 blocks (round up) use poora cover kar dete hain — thodi si overshoot, jo theek hai; overshoot kuch waste nahi karta, undershoot cycles waste karta hai.

Step 7 — Degenerate cases (reader ko kabhi stranded mat chhodna)

Case A — koi memory reads nahi ( $L = 0$ ): $N = ⌈ 0/ W ⌉ = 0$ extra warps chahiye. Ek warp jo kabhi wait nahi karta use cover karne ke liye kisi ki zaroorat nahi. ✔ (Compute-bound kernel.)

Case B — saara kaam, chhote gaps ( $W$ bada): $\frac{L}{W} \to$ ek chhota number, ceiling 1 deta hai. Ek warp apni latency khud almost hide kar leta hai instruction-level parallelism se. ✔

Case C — reads ke beech koi kaam nahi ( $W = 0$ ): division by zero — $\frac{L}{0}$ undefined hai, aur hona bhi chahiye: ek warp jo reads ke beech zero kaam karta hai kabhi self-cover nahi kar sakta; aapko infinitely many warps chahiye honge. Practice mein hardware ~64 resident warps par cap karta hai, toh aap simply limit saturate karte ho aur phir bhi stall karte ho. ✔ Yeh memory-bound wall hai.

PICTURE: Teen mini-timelines side by side: $L = 0$ (saab kala, koi red nahi), $W$ bada (ek warp, red mostly apne kale se fill), $W = 0$ (endless red, koi bhi kala kabhi fill nahi kar sakta).

Ek-picture summary

Upar sab kuch ek single frame mein: warp 0 ka $L$ -cycle red hole, exactly $⌈ L / W ⌉$ kale work-blocks doosre warps se edge-to-edge tile kiye gaye, aur neeche formula stamped.

Recall Feynman retelling — ise simple words mein wapas bolo

Ek warp 32 workers hain jo har step saath lene chahiye. Saste steps ek tick mein hote hain; door memory se koi number maangna $L$ ticks ka bada stall leta hai. Agar ek warp sirf woh stall wait karta, machine ~98% waqt idle rehti — ek disaster.

Toh scheduler kuch smart karta hai: jis pal ek warp stall kare, woh ek doosra warp idle gap mein slide karta hai aur uska kaam run karta hai — free mein, kyunki har warp apne registers aur apna bookmark rakhta hai, toh switch karna ek pointer update hai.

Har substitute warp $L$ -tick hole ke $W$ ticks fill karta hai. " $L$ -hole mein kitne $W$ -chunks fill hote hain?" sirf $L$ divided by $W$ hai. Kyunki ek bacha hua sliver idle time matlab hai, hum up round karte hain — the ceiling. Yahi poori kahani hai: $N_{warps needed} = ⌈ \frac{L}{W} ⌉ .$ Edge checks: koi memory reads nahi → kisi ki zaroorat nahi; read per bada kaam → ek chahiye; read per zero kaam → koi bhi warps ka number bachaa nahi sakta (memory-bound). Read per zyaada compute matlab kam warps chahiye — isliye pure occupancy ke peeche bhaagna aapko galat direction mein le ja sakta hai.

Recall Quick self-test

$L$ kya measure karta hai? ::: Ek memory read ke liye wait kiye cycles (latency), ~400. $W$ kya measure karta hai? ::: Useful work-cycles jo ek warp memory reads ke beech run karta hai. Ceiling kyun, floor nahi? ::: Neeche round karne se idle sliver bachta hai; ceiling guarantee karta hai hole poora cover ho. $L = 400, W = 8$ ke saath kitne warps? ::: $⌈ 400/8 ⌉ = 50$ . Warps switch karna free kyun hai? ::: Har warp apne registers + program counter rakhta hai, toh switch ek pointer update hai (zero-overhead). Kya hota hai jab $W = 0$ ? ::: Undefined / infinite — ek warp jiske reads ke beech koi compute nahi woh kabhi self-cover nahi kar sakta; aap memory-bound ho jaate ho.