Forward propagation computation — Hinglish notes

3.1.3 · AI-ML › Neural Network Fundamentals

Forward propagation exist kyun karta hai?

Kaunsa problem solve karta hai? Ek neural network ek function $\overset{y}{^} = f (x)$ hai jisme millions tunable knobs (weights) hote hain. Jab tak hum un knobs ko seekhna (backprop) shuru karein, pehle hume diye gaye input ke liye prediction compute karni aani chahiye. Forward propagation wohi computation hai — "network run karo" wala step.

Yeh design (linear + nonlinear) kyun?

Pure stack of linear layers collapse ho jaati hai: $W_{2} (W_{1} x) = (W_{2} W_{1}) x$ — kitni bhi deep ho, phir bhi linear rehti hai. Isliye depth useless ho jaati.
Layers ke beech mein ek nonlinear activation daalna us collapse ko todta hai, jisse network curved, complex functions ko approximate kar sake. Yehi reason hai ki deep nets itne powerful hote hain.

HOW: pehle principles se build karna

Step 1 — Ek single neuron

Ek neuron inputs $x_{1}, \dots, x_{n}$ receive karta hai, jinme har ek ki importance $w_{i}$ hoti hai, plus ek bias $b$ .

Bias kyun? $b$ ke bina, har neuron ki decision boundary origin se guzarni chahiye. Bias shift karta hai activation ko left/right, taaki neuron inputs zero hone par bhi fire kar sake.

Step 2 — Ek layer ko vectorize karo

Ek layer mein $m$ neurons stack karke, jinme se har ek ki apni weight row hoti hai. Unhe collect karo:

z = W x + b, a = ϕ (z)

$W$ ki row $j$ mein neuron $j$ ke weights kyun hote hain? Kyunki matrix–vector multiply har output ko ek row aur input ke dot product se compute karta hai. Row $j$ · input $= z_{j}$ — bilkul wohi weighted sum jo neuron $j$ ke liye chahiye.

Step 3 — Poore network ko chain karo

Input $a^{[0]} = x$ ke saath, $ℓ = 1, 2, \dots, L$ ke liye repeat karo:

$a^{[0]} = x ⟶ z^{[1]}, a^{[1]} ⟶ \dots ⟶ a^{[L]} = \overset{y}{^}$

Final activation $a^{[L]}$ prediction hai. Classification ke liye $ϕ^{[L]}$ usually softmax hoti hai; regression ke liye aksar identity hoti hai.

Figure — Forward propagation computation

Step 4 — Batch form (matrices kyun use karte hain)

Ek input vector ki jagah, $N$ examples ek saath $X$ ( $n \times N$ ) ke columns ke roop mein process karo: $Z^{[ℓ]} = W^{[ℓ]} A^{[ℓ - 1]} + b^{[ℓ]}, A^{[ℓ]} = ϕ (Z^{[ℓ]})$ Bias $b^{[ℓ]}$ ( $m \times 1$ ) saare $N$ columns mein broadcast hoti hai. Batch kyun? Ek bada matmul hardware (GPU) ko $N$ chhote matmuls se kahin zyada efficiently use karta hai.

Common activations (aur WHY har ek)

Common hidden activations: ReLU $ϕ (z) = max (0, z)$ (sasta, $z > 0$ ke liye vanishing gradient nahi), sigmoid $ϕ (z) = \frac{1}{1 + e ^{- z}}$ ( $(0, 1)$ mein squash karta hai), tanh ( $(- 1, 1)$ mein squash karta hai).

Worked Example 1 — haath se ek tiny 2-layer network

Input $x = [12]$ . Hidden layer (2 units), ReLU: $W^{[1]} = [0.5 1 - 0.5 1], b^{[1]} = [0 - 1]$ Output layer (1 unit), identity: $W^{[2]} = [2 - 1], b^{[2]} = 0.5$

Layer 1 pre-activation — Kyun? Inputs ko weights + bias se combine karo: $z^{[1]} = [0.5 (1) + (- 0.5) (2) + 0 1 (1) + 1 (2) - 1] = [- 0.5 2]$

Layer 1 activation — Kyun? ReLU apply karo (negatives ko 0 pe clip karo): $a^{[1]} = [max (0, - 0.5) max (0, 2)] = [02]$

Layer 2 — Kyun? Same recipe, identity activation: $z^{[2]} = 2 (0) + (- 1) (2) + 0.5 = - 1.5, \overset{y}{^} = - 1.5$

Worked Example 2 — softmax output

Logits $z^{[L]} = 210$ . Exponentiate kyun? Scores ko positive, comparable weights mein convert karne ke liye: $e^{2} = 7.389, e^{1} = 2.718, e^{0} = 1, sum = 11.107$ $\overset{y}{^} = 0.665 0.245 0.090$ Sum se divide kyun? Taaki outputs ek probability distribution banayein ( $\sum = 1$ ). Class 0 sabse zyada likely hai.

Forecast-then-Verify

Common mistakes (Steel-manned)

Recall Ek 12-saal ke bache ko samjhao (Feynman)

Ek bucket-brigade socho jo paani pass kar rahi ho. Har insaan apne peeche waalon se paani leta hai, lekin kuch padosiyon par zyada bharosa karta hai — isliye trusted waalon se zyada scoop karta hai (ye hain weights). Woh apna ek fixed splash bhi milata hai (bias). Phir ise ek ajeeb funnel se dalte hain jo kitna baahir nikalta hai use change karta hai (activation). Paani insaan dar insaan aage pass hota rehta hai jab tak aakhri insaan answer nahi dikhaata. Forward propagation bas ek baar is brigade se aage ki taraf paani ka bahna hai.

Flashcards

Ek layer ke forward pass mein kaun se do operations hote hain?

Ek linear step

z = W a^{[ℓ - 1]} + b

, phir ek nonlinear activation

a = ϕ (z)

Layers mein nonlinear activation kyun zaroori hai?

Warna stacked linear layers ek single linear map mein collapse ho jaati hain, isliye depth koi expressive power nahi add karti.

n

inputs aur

m

units wali layer ke liye

W

ki shape kya hogi?

m \times n

(output rows × input columns), taaki

W a

m

-vector de.

Bias term geometrically kya karta hai?

Yeh pre-activation ko shift karta hai, taaki ek neuron inputs zero hone par bhi fire kar sake (decision boundary ka origin se guzarna zaroori nahi).

Softmax mein exponentials kyun use karte hain?

Ye positive aur monotonic hote hain, isliye bade logits → badi probabilities; unka sum se divide karna outputs ko ek valid distribution banata hai jiska sum 1 ho.

Pre-activation

z

kya hai?

Weighted sum plus bias,

z = \sum_{i} w_{i} x_{i} + b

, activation apply karne se pehle.

Batch forward prop mein bias broadcast kyun hoti hai?

Wohi

m \times 1

bias

Z = W A + b

ke saare

N

example-columns par apply hoti hai.

Classifier ke output ke liye kaunsi activation typical hai aur regressor ke liye kaunsi?

Multi-class classification ke liye softmax; regression ke liye identity (linear).

Batched matrix multiplies per-sample loops se zyada prefer kyun ki jaati hain?

Ek bada matmul GPU/BLAS hardware ko bahut saare chhote matmuls se kahin zyada efficiently use karta hai.

Connections

Activation functions — nonlinearities $ϕ$ jo forward prop ko expressive banati hain.
Backpropagation — forward prop se cached $z, a$ reuse karke gradients compute karta hai.
Matrix multiplication — har layer ka computational engine.
Softmax and cross-entropy loss — output activation + prediction loss kaise banta hai.
Universal approximation theorem — kyun ek nonlinear hidden layer bhi bahut badi power deti hai.
Vanishing and exploding gradients — yahan ki gayi activation choices ka downstream consequence.