Foundations — Wafer-scale engines (Cerebras-style)

6.5.17 · D1 · Hardware › Advanced & Emerging Architectures › Wafer-scale engines (Cerebras-style)

Is page mein kuch bhi assumed nahi hai. Agar parent note mein koi symbol likha tha, toh hum use yahaan pehle picture se build karte hain, phir batate hain ki topic ko wo kyun chahiye. Upar se neeche padho: har idea agli ke liye ek brick hai.

1. Wafer, die, aur dicing streets

Picture: figure dekho. Circle wafer hai. Blue squares ki grid dies hain. Unke beech gray lanes scribe lines hain. Ek normal factory un gray lanes par kaatti hai aur blue squares alag alag bechti hai.

Topic ko yeh kyun chahiye: WSE ka poora trick hai ki kaat mat karo aur gray lanes ke paas metal wiring add karo taaki saare blue squares ek connected chip ban jayein. Yeh sentence in char words ke bina samajh nahi aata.

2. Area, aur "kitna bada" ek number kyun hai ($A$)

Picture: upar ki figure mein, $A$ ek blue square ka size hai. Ek whole-wafer die ek enormous square hai jo poori plate bhar deta hai — roughly $A\approx 46,000{\text{mm}}^2$.

Hamen yeh kyun chahiye: bada $A$ matlab chip wafer ka zyada hissa cover karti hai, matlab ek random flaw swallow karne ka chance zyada hai. Area woh knob hai jo yield ko "easy" se "impossible" karta hai. Yeh thought pakad ke rakho — hum ise §5 mein exact karte hain.

3. Defects aur defect density ($D$)

Picture: wafer par bikhe red dots defects hain. Yeh random jagahon par girte hain, independently — ek defect ko nahi pata baaki kahan hain. Kisi bhi $1{\text{mm}}^2$ box mein dots gino, poore wafer par average lo, aur woh average $D$ hai.

Hamen yeh kyun chahiye: $D$ batata hai wafer kitna hostile hai. Ek chota die shayad saare dots se bach jaaye; ek giant die almost certainly kai pakad leta hai. "Almost certainly" ko real probability mein badalne ke liye hamen ek mathematical tool chahiye — Poisson distribution — aage aata hai.

4. Randomness jo add hoti hai: Poisson idea ($\lambda$, mean)

Yahi tool kyun? Humare paas random dots (§3) hain jo ek area (§2) par independently bikre hain. "Independent rare events scattered over space or time" ke liye precisely banaya hua ek probability recipe hai Poisson distribution. Hum bell curve ya coin-flip formula nahi lete kyunki wo alag situations describe karte hain; Poisson woh hai jo jawaab deta hai "average count $\lambda$ diya, $k$ events dekhne ka chance kya hai?"

Yahaan yeh matter kyun karta hai: ek die kaam karne ke liye usme zero killing defects chahiye, matlab $k=0$. Wafer yield ki poori baat ek sawaal par aa jaati hai: $P(0)$ kya hai? Hum iska jawaab $e$ se milne ke baad dete hain.

5. Number $e$ aur exponential $e^{-x}$

Picture: figure mein curve $e^{-x}$ hai. $x=0$ par yeh $1$ hai (100% — certain). $x=1$ tak yeh $\approx 0.37$ hai. $x=5$ tak yeh $1\%$ se neeche hai. Yeh cliff se gir jaata hai.

Yeh tool kyun? Poisson formula mein $k=0$ daalo: $P(0)=\frac{{\lambda }^0{e}^{-\lambda }}{0!}=\frac{1\cdot e^{-\lambda }}{1}=e^{-\lambda }.$ Poori complicated formula ek clean exponential mein collapse ho jaati hai. Aur $\lambda$ — ek die mein average defects — bas density times area hai: $\lambda =D\cdot A$. Toh ek die perfect hone ka probability hai:

Picture ko physics ki tarah padhna: jaise jaise $A$ poore wafer ki taraf badhta hai, $DA$ ek bada number ban jaata hai, aur $e^{-x}$ curve batata hai ki $Y$ $0$ ki taraf dive karta hai. Yahi mathematical reason hai ki monolithic whole-wafer chip "kabhi kaam nahi karni chahiye" — aur isliye topic ko isse bachne ke liye redundancy chahiye.

6. Fractions, thresholds, aur over-provisioning ($f$, $N$, $M$)

Picture: socho $M$ light bulbs build kar rahe ho jaante hue ki fraction $f$ duds honge. Working bulbs $=M(1-f)$. Kam se kam $N$ jale hue paane ke liye, extra build karna hoga: $M(1-f)\ge N\Rightarrow M\ge \frac{N}{1-f}.$

Topic ko yeh kyun chahiye: WSEs perfect wafer dhundhna band kar dete hain (impossible, §5) aur instead over-provision karte hain — $\frac{1}{1-f}$ factor se spare cores build karo aur dead walo ke around route karo. Yeh akela inequality hi wafer-scale ko buildable banata hai. Dekho Yield & Defect Density Models.

7. Cores, SRAM, routers, aur mesh

Picture: squares ki grid mesh hai. Har square mein ek core + uska SRAM + ek router hai. Arrows sirf touching neighbours ko link karte hain — koi long wires grid ke paas jump nahi karti. Ek far corner se message square se square hop karta hai.

Topic ko yeh kyun chahiye: mesh WSE ki dono strengths explain karta hai — high bandwidth (saare links ek saath chalte hain, §8) aur nearby traffic ke liye low latency — aur iska weakness bhi (door ke cores ko zyada hops chahiye). Yeh Network-on-Chip (NoC) & Mesh Topologies idea hai, aur isliye weight-stationary Systolic Arrays & TPUs-style dataflow (har core apne weights rakhta hai, activations stream hoti hain) itna fit baithta hai.

8. Bandwidth, latency, aur kyun yeh add hote hain ($B$, hops)

Kyun yeh add hote hain (key move): ek mesh mein, links parallel mein chalte hain — jab ek core left bhejta hai, hazaron doosre bhi bhejte hain, sab ek saath. Toh total bandwidth sum hai: $B_{total}=M\cdot B_{core}.$ $M=850,000$ cores ke saath, har ek $B_{core}=1\text{GB/s}$ push karte hue: $B_{total}=850,000\times 1\text{GB/s}=850,000\text{GB/s}=850\text{TB/s}.$ (Yahaan $1\text{TB/s}=1000\text{GB/s}$.)

Topic ko dono kyun chahiye: parent ka poora argument yeh hai ki off-chip links time aur energy waste karte hain. Bandwidth dikhata hai mesh GPU link se ~1000× zyada data move karta hai; latency dikhata hai ek neighbour hop ~1 ns ka hai versus ek network ke across microseconds. Yeh Dennard Scaling & the Memory Wall ka doosra side hai — compute badha, off-chip data movement nahi badhaya.

9. Yeh pieces wafer ko abhi bhi kahaan chhodte hain

Prerequisite map

Yeh map seedha Wafer-scale engines (Cerebras-style) mein feed karta hai. Related integration ideas Chiplets & 2.5D/3D Integration mein compare karo, aur Hinglish version padho agar woh help kare.

Equipment checklist

Right side cover karo aur reveal karne se pehle zor se jawaab do.