Visual walkthrough — Feedforward network — forward pass

5.6.7 · D2 · Coding › Machine Learning (Aerospace Applications) › Feedforward network — forward pass

---

Step 1 — Ek neuron bas ek dot hai, jo ek number hold karta hai

KYA HAI. Ek single circle banao. Uske andar ek number rehta hai. Woh hai ek neuron. Jo number woh hold karta hai use activation kehte hain — sochlo yeh kitni strongly yeh neuron fire kar raha hai, jaise ek dimmer switch "off" aur "bahut on" ke beech mein.

KYO. Kisi bhi math se pehle, hume sabse chhote object pe agree karna hoga. Baaki sab cheez — layers, matrices, poora network — is ek circle ki copies hain jo ek doosre se wired hain. Agar aap circle ko samajh gaye, toh network samajh gaye.

PICTURE. Figure dekho. Ek cyan circle, andar ek amber number. Arrows jo andar aate hain woh wires hain jo ise feed karengi; bahar jaane wala arrow uski activation aage carry karta hai.

---

Step 2 — Ek wire multiply karta hai; weight us wire ka volume knob hai

KYA HAI. Do circles ko ek wire se connect karo. Baayein circle mein activation $a_j^{(\ell -1)}$ hai. Wire woh number daayein circle tak le jaata hai, lekin pehle use scale karta hai ek number se jo weight $w_{ij}^{(\ell )}$ kehlata hai. Doosri taraf jo pahunchta hai woh product hai $w_{ij}^{(\ell )}{a}_j^{(\ell -1)}$.

KYO. Ek raw sensor reading (maano Mach 0.8) alag-alag decisions ke liye alag meaning rakhti hai. Weight hai yeh decision us reading ki kitni parwah karta hai. Bada positive weight = "dhyan se suno aur agree karo." Negative weight = "suno, lekin ulta karo." Zero = "is wire ko ignore karo." Multiplication natural tool hai kyunki input ko double karna uska influence double karna chahiye — yeh exactly wohi hai jo ek fixed number se multiply karne pe hota hai.

PICTURE. Figure mein input number dikha hai, weight wire pe baitha hai jaise volume knob, aur scaled number bahar aa raha hai. Amber label ka size change hota dekho.

---

Step 3 — Ek neuron apni saari wires add karta hai (weighted sum)

KYA HAI. Ek neuron mein usually bahut saari wires aati hain. Woh simply har scaled number ko add up karta hai: $\underset{\text{saare input neurons }j\text{ pe add karo}}{\underbrace{\sum _{j=1}^{n_{\ell -1}}}}\underset{\text{wire }j\text{ ka knob}}{\underbrace{w_{ij}^{(\ell )}}}\underset{\text{neuron }j\text{ ne kya bheja}}{\underbrace{a_j^{(\ell -1)}}}$

KYO. Har wire ek evidence ka tukda hai. Unhe sum karna hai woh tarika jisse neuron evidence ko ek single score mein pool karta hai: strong-agree wires sum ko upar push karte hain, disagree wires (negative weights) ise neeche pull karti hain. Addition sahi tool hai kyunki evidence accumulate hona chahiye — do aadhe reasons milke ek poora reason banna chahiye.

PICTURE. Teen input circles, teen knobs, teen arrows sab ek bade plus-sign mein funnel ho rahe hain, ek running total de rahe hain. $\sum$ (capital Greek "S," Sum ke liye) sirf shorthand hai "inhe sab add karo," jahan $j$ pehle input neuron se last tak count karta hai, $n_{\ell -1}$ (previous layer mein neurons ki sankhya).

---

Step 4 — Bias add karo: neuron ki apni built-in opinion

KYA HAI. Wires sum karne ke baad, neuron ek aur number add karta hai jo kisi bhi input pe depend nahi karta — bias $b_i^{(\ell )}$. Poora total hai pre-activation: $z_i^{(\ell )}=\underset{\text{wires se evidence}}{\underbrace{\sum _j{w}_{ij}^{(\ell )}{a}_j^{(\ell -1)}}}+\underset{\text{fixed offset}}{\underbrace{b_i^{(\ell )}}}$

KYO. Kabhi kabhi ek neuron ko fire karna chahiye chahe saare inputs quiet ho, ya refuse karna chahiye fire karne se jab tak evidence overwhelming na ho. Bias poore total ko upar ya neeche shift karta hai, neuron ka "default mood" set karta hai. Iske bina, har neuron ko jab input sab zeros ho toh same cheez output karne pe majboor hona padta — bahut rigid.

PICTURE. Step 3 ka plus-sign total, phir ek chhota amber $+b$ box bar ko upar nudge karta hai (positive bias) ya neeche (negative bias) pehle ki hum $z$ padhein.

---

Step 5 — Wires ko ek matrix multiply mein stack karo

KYA HAI. Layer $\ell$ mein har neuron apna weighted sum karta hai. Har neuron ke liye alag equation likhne ki bajaye, hum saare neurons ki activations ko numbers ke ek column mein (vector) aur saare weights ko ek grid mein (matrix) stack karte hain. Phir poori layer ek line mein: $\underset{\text{column, ek entry per neuron}}{\underbrace{{\mathbf{z}}^{(\ell )}}}=\underset{\text{saare weights ka grid}}{\underbrace{{\mathbf{W}}^{(\ell )}}}\underset{\text{previous layer ka column}}{\underbrace{{\mathbf{a}}^{(\ell -1)}}}+\underset{\text{biases ka column}}{\underbrace{{\mathbf{b}}^{(\ell )}}}$

KYO. Bold symbols sirf bahut saare numbers bundle karte hain taaki hum saikdon ki jagah ek clean line likhein. Ek matrix–vector product defined hi aise hai ki "har output = matrix ki ek row ka input vector ke saath dot product" — jo precisely Steps 3–4 ka weighted sum hai. Toh yeh compact line kuch hide nahi karti; woh wohi arithmetic hai. Computers bhi matrices bahut fast multiply karte hain, isliye yeh form hi actually aircraft pe run hoti hai.

PICTURE. Grid ${\mathbf{W}}^{(\ell )}$ ki row $i$ light up hoti hai, input column ke across slide karti hai, multiply-and-add karti hai, aur ${\mathbf{z}}^{(\ell )}$ ke slot $i$ mein ek number drop karti hai. Grid mein $n_{\ell }$ rows hain (is layer mein ek per neuron) aur $n_{\ell -1}$ columns (ise feed karne wale ek per neuron).

---

Step 6 — Line ko bend karo: activation function

KYA HAI. Raw tally ${\mathbf{z}}^{(\ell )}$ ko ek bending function ${\sigma }^{(\ell )}$ se feed karo, jo har entry pe apne aap apply hoti hai: ${\mathbf{a}}^{(\ell )}={\sigma }^{(\ell )}\left({\mathbf{z}}^{(\ell )}\right)$ Common choices: ReLU $\sigma (z)=\max (0,z)$ (kuch bhi negative flatten karke 0 kar do), aur tanh (koi bhi number squash karke $-1$ se $+1$ range mein).

YEH TOOL KYO aur plain addition nahi? Kyunki agar har layer sirf sums-aur-scales ho (straight-line maths), toh layers stack karna ek single straight-line map mein collapse ho jaata — chahe kitni bhi layers stack karo. Neeche proof-by-picture hai: do straight lines compose karke bhi ek straight line rehti hai. Ek curved function hi ek maatra tarika hai aisi shapes banane ka jo straight lines nahi bana sakti — aur stall, shock waves, aur drag curves sab bendy hain. Bend hi woh jagah hai jahan se "learning power" aati hai.

PICTURE. Left panel: ReLU elbow — left pe flat, phir $45^{\circ }$ ramp; negative $z$ $0$ pe land karta hai, positive $z$ unchanged pass ho jaata hai. Right panel: tanh S-curve edges ke paas $\pm 1$ ki taraf flatten hoti hui.

---

Step 7 — Bend optional kyon nahi hai (degenerate case)

KYA HAI. Maano hum har activation hata dein — har $\sigma$ ko "kuch mat karo" set kar dein. Phir layer ke baad layer sirf multiply-aur-add hai. Do chain karo: ${\mathbf{a}}^{(2)}={\mathbf{W}}^{(2)}({\mathbf{W}}^{(1)}\mathbf{x}+{\mathbf{b}}^{(1)})+{\mathbf{b}}^{(2)}=\underset{\text{ek naya grid }{\mathbf{W}}_{\text{eff}}}{\underbrace{{\mathbf{W}}^{(2)}{\mathbf{W}}^{(1)}}}\mathbf{x}+\underset{\text{ek naya offset }{\mathbf{b}}_{\text{eff}}}{\underbrace{{\mathbf{W}}^{(2)}{\mathbf{b}}^{(1)}+{\mathbf{b}}^{(2)}}}$

KYO DIKHANA HAI. Yeh pehle ke claim ko visually aur algebraically prove karta hai: ek 100-layer linear network ek 1-layer network ${\mathbf{W}}_{\text{eff}}\mathbf{x}+{\mathbf{b}}_{\text{eff}}$ ke barabar hai. Itni depth ne kuch nahi kharida. Yeh woh degenerate case hai jisme reader ko kabhi nahi jaana chahiye.

PICTURE. Upar: do straight-line transforms do rulers ke roop mein draw ki gayi; neeche, woh ek single ruler mein merge ho jaati hain — same output, ek step. Amber caption: "depth wasted."

---

Step 8 — Layers ko wire karo: poora forward pass

KYA HAI. Shuru karo input ko layer 0 ki activation ke roop mein rename karke, taaki har layer identical dikhe: ${\mathbf{a}}^{(0)}=\mathbf{x}$ Phir Steps 5–6 ko $\ell =1,2,\dots ,L$ ke liye repeat karo, har layer ka output agli mein feed karte hue: ${\mathbf{a}}^{(\ell )}={\sigma }^{(\ell )}({\mathbf{W}}^{(\ell )}{\mathbf{a}}^{(\ell -1)}+{\mathbf{b}}^{(\ell )}),\mathbf{y}={\mathbf{a}}^{(L)}$

INPUT RENAME KYO KARO? Taaki wohi recipe har layer pe chale — pehli ke liye koi special case nahi. Uniformity code ko ek clean loop aur maths ko ek clean recursion banata hai. Hum layer $L$ (last wali) pe rukते hain; uski activation hi prediction $\mathbf{y}$ hai.

PICTURE. Poora pipeline left-to-right: input column → grid → bend → column → grid → bend → output. Data sirf rightward flow karta hai — yeh ek-taraf ka flow hi hai jiske kaaran ise "forward" kehte hain.

---

Step 9 — Parent ke numbers flow hote dekho (2-3-1 case)

KYA HAI. Parent note ke Example 1 ko hamare picture pipeline se run karo. Input $\mathbf{x}=\left[\begin{array}{c}0.8\\ 10000\end{array}\right]$, ek hidden layer (3 neurons, ReLU), ek output (linear).

• Layer 1 tally: ${\mathbf{z}}^{(1)}=\left[\begin{array}{c}1.5\\ 0.06\\ -0.84\end{array}\right]$ → ReLU ke baad: ${\mathbf{a}}^{(1)}=\left[\begin{array}{c}1.5\\ 0.06\\ 0\end{array}\right]$ (neuron 3 flatten hokar 0 — is input ke liye ek dead neuron).
• Output tally: $z^{(2)}=1.0(1.5)-0.5(0.06)+2.0(0)+0.3=1.77$, linear → $\mathbf{y}=1.77$.

YEH YAHAN KYUN REVISIT KARTE HAIN. Ab har number ki ek picture hai: ReLU elbow neuron 3 ko clip karta hua, output dot-product jo survived items ko sum karta hua. Abstract recursion ek concrete waterfall of numbers ban jaati hai.

PICTURE. Har neuron ek circle ke roop mein draw kiya gaya jiska size uski activation ke hisaab se hai; neuron 3 greyed-out (dead); final amber $1.77$ output pe glow kar raha hai.

---

Ek-picture summary

Upar ki saari cheez ek single blueprint mein compress ki gayi: input column left se enter karta hai, har layer pe ek grid (matrix) weighted sums karta hai, ek plus bias add karta hai, ek bend ($\sigma$) result ko curve karta hai, aur arrow strictly rightward march karta rehta hai jab tak amber output pop out na ho jaaye. Yahi poora forward pass hai.

Recall Feynman retelling — plain words mein khud bolo

Ek neuron ek dot hai jo ek number hold karta hai. Wires numbers laate hain pehle wali layer se, har wire apne number ko ek weight se scale karti hai (ek volume knob). Neuron un saare scaled numbers ko add karta hai, phir apna bias add karta hai — yahi raw tally $z$ hai. Phir woh tally ko ek curved function se bend karta hai (ReLU negatives flatten karta hai; tanh $-\mathrm{1..1}$ mein squash karta hai) apni activation paane ke liye. Inki ek poori layer stack karna hai ek matrix-multiply-plus-bias-then-bend. Hum input ko "layer 0" rename karte hain, phir recipe layer by layer repeat karte hain, hamesha forward feed karte hue, kabhi backward nahi — yeh ek-taraf ki march hi iska naam forward pass rakhti hai. Bend mandatory hai: iske bina, sau layers ek straight line mein collapse ho jaati hain aur saari depth waste ho jaati hai. Hum last layer pe rukते hain; uski activation prediction hai.

Recall

Network ko "feedforward" kyon kehte hain? ::: Kyunki activations sirf ek direction mein flow karti hain, input → output, kabhi loop nahi karti. Weight $w_{ij}^{(\ell )}$ ek wire ke saath kya karta hai? ::: Woh neuron $j$ se neuron $i$ tak travel karne wale number ko multiply (scale) karta hai — ek volume knob. Pre-activation $z_i^{(\ell )}$ kya hai? ::: Inputs ka weighted sum plus bias, activation function apply hone se pehle. Activation function nonlinear kyon honi chahiye? ::: Warna stacked layers ek single affine map ${\mathbf{W}}_{\text{eff}}\mathbf{x}+{\mathbf{b}}_{\text{eff}}$ mein collapse ho jaati hain, toh depth kuch nahi khareeda. $-0.84$ ka ReLU kya hai? ::: $0$ — ReLU har negative ko zero flatten kar deta hai. Example 1 ke network ka output kya hai? ::: $1.77$.