Feedforward network — forward pass: Question bank

5.6.7 · D5 · Coding › Machine Learning (Aerospace Applications) › Feedforward network — forward pass

Shuru karne se pehle, teen words ka matlab tere liye bilkul wahi hona chahiye jo parent note mein hai, toh main unhe plain language mein pin kar deta hoon:

True or false — justify

Nonlinearity ke claims woh jagah hain jahan zyaadatar misconceptions rehti hain, toh hum wahan se shuru karte hain.

Do linear (identity-activation) layers stack karke kuch aisa represent kar sakte hain jo ek single linear layer nahi kar sakti

False. Do affine maps compose karne se milta hai

W_{2} (W_{1} x + b_{1}) + b_{2} = (W_{2} W_{1}) x + (W_{2} b_{1} + b_{2})

, jo phir se ek affine map

W_{eff} x + b_{eff}

hai. Koi nayi power nahi milti.

Bias vector

b^{(ℓ)}

hi network ko nonlinear banata hai

False. Bias sirf weighted sum ko upar ya neeche shift karta hai; ek seedhi line ko shift karne se woh seedhi hi rehti hai. Nonlinearity poori tarah activation function

σ

se aati hai.

Forward pass weights ko run hone ke dauran change karta hai

False. Forward pass weights aur biases ko fixed constants ki tarah padhta hai aur sirf activations produce karta hai. Weights training ke dauran backpropagation ke zariye change hote hain, jo ek alag, backward step hai.

a^{(0)} = x

ka matlab hai ki layer 0 sirf input ka ek relabelling hai jisme koi computation nahi

True. Hum raw input ko "layer 0 ki activation" isliye kehte hain taaki recursion

z^{(ℓ)} = W^{(ℓ)} a^{(ℓ - 1)} + b^{(ℓ)}

ka ek uniform starting point ho. Layer 0 ko koi weights touch nahi karte.

Kyunki information sirf input→output flow karti hai, ek feedforward net mein phir bhi ek loop ho sakta hai jab tak data kabhi backward nahi travel karta

False. "Feedforward" ka matlab hai computation graph ek directed acyclic graph hai: bilkul bhi koi cycle nahi. Ek loop "one pass" ko undefined bana deta — tumhe kabhi pata nahi chalta kab rukna hai.

ReLU ko element-wise apply karna aur ise pore vector par ek saath apply karna alag results dete hain

False. ReLU (aur tanh, sigmoid) har entry par independently act karte hain, toh

σ (z)

exactly woh vector hai jisme

σ

har component par apply hoti hai. Ek activation ke andar entries ke beech koi cross-talk nahi hoti.

Softmax ek element-wise activation hai bilkul ReLU ki tarah

False. Softmax ek common exception hai: har output saari pre-activations par depend karta hai (yeh har

e^{z_{i}}

ko sum

\sum_{k} e^{z_{k}}

se divide karta hai). Yahi coupling outputs ko 1 mein sum karne aur probabilities ki tarah behave karne par majboor karti hai.

Spot the error

Har item mein kuch aisa hai jo ek student sochta hai sahi hai. Galti batao.

W^{(ℓ)}

ka shape

(n_{ℓ - 1}, n_{ℓ})

hai toh

a^{(ℓ - 1)}

ke saath product kaam karta hai."

Wrong shape. Yeh

(n_{ℓ}, n_{ℓ - 1})

hona chahiye: rows = current layer ke neurons, columns = feed karne wale neurons. Tabhi

W^{(ℓ)} a^{(ℓ - 1)}

n_{ℓ}

-vector deta hai.

"2-3-1 net ke liye,

a^{(1)}

ek 2-vector hai kyunki input mein 2 features the."

Wrong dimension. Hidden layer mein 3 neurons hain, toh

a^{(1)} \in R^{3}

. Activation size current layer ke neuron count se set hoti hai, input se nahi.

"Neuron 3 ne Example 1 mein 0 output kiya, toh hum us neuron ko delete kar sakte hain; yeh kuch contribute nahi karta."

Case error. Usne us ek input ke liye 0 output kiya kyunki uska pre-activation negative tha (

- 0.84

). Kisi alag input ke liye wahi neuron strongly fire kar sakta hai. ReLU zeros input-dependent hain, permanent nahi.

"Maine hidden layer par softmax apply kiya probabilities paane ke liye, phir unhe aage feed kiya."

Wrong layer. Softmax ek classifier ke output par hona chahiye taaki final scores class probabilities ki tarah padhe jaayein. Ise internally use karna hidden features ko squash aur normalise kar deta hai, woh raw signal destroy kar deta hai jo next layer expect karti hai.

z^{(2)} = a^{(1)} W^{(2)} + b^{(2)}

— matrix times vector, order matter nahi karta."

Order matters. Yeh

W^{(2)} a^{(1)} + b^{(2)}

hai: matrix left par hai. Matrix multiplication commutative nahi hai, aur

a^{(1)} W^{(2)}

generally ek legal shape bhi nahi hota.

"Output prediction

z^{(L)}

hai, last layer ka pre-activation."

Off by one step. Prediction

y = a^{(L)} = σ^{(L)} (z^{(L)})

hai. Tumhe abhi bhi final activation se guzarna hoga — chahe

σ^{(L)}

identity hi kyun na ho, woh identity choice ek decision hai, skip nahi.

"Maine inputs normalise kar diye, toh mujhe altitude column par chote weights ki zarurat nahi rahi."

Confused cause. Raw example mein tiny altitude weights un-normalised data ke liye ek patch the. Jab inputs normalise ho jaate hain toh network ordinary weights seekh sakta hai — dono same scale problem ke alternative fixes hain, dono ki zarurat nahi.

Why questions

Reasoning explain karo; ek bare fact kaafi nahi hai.

Hum activations layer by layer kyun compute karte hain instead of seedha output par jump karne ke?

Kyunki layer

ℓ

ka input hi layer

ℓ - 1

ka output hai — computation ek chain of dependencies hai. Jab tak

a^{(ℓ - 1)}

exist na kare, physically

z^{(ℓ)}

form nahi ho sakta.

Parent note input ko sirf "

x

" ki jagah "

a^{(0)}

" kyun kehta hai?

Recurrence formula ko uniform banane ke liye: ek rule

z^{(ℓ)} = W^{(ℓ)} a^{(ℓ - 1)} + b^{(ℓ)}

ab har layer cover karta hai, pehli layer bhi, bina kisi special case ke.

ReLU ko "sparse" representations create karne wala kyun kaha jaata hai?

Kyunki yeh har negative pre-activation ko exactly 0 kar deta hai, typically bahut saare neurons ko ek saath silence kar deta hai. Bahut saare zeros wala vector sparse hota hai, jisse har active neuron ka role interpret karna aasaan ho jaata hai.

Aerospace models nonlinear kyun hone chahiye — ek bade linear net par trust kyun nahi?

Stall, shock waves aur compressibility jaise phenomena genuinely curved relationships hain. Ek linear (affine) map sirf seedhi lines ko tilt aur shift kar sakta hai, toh chahe tum kitne bhi linear layers stack karo woh un curves ko fit karne ke liye bend nahi kar sakta.

Thrust regression ke liye linear output activation kyun suit karta hai par fault classification ke liye nahi?

Regression ko ek unbounded real number chahiye (thrust koi bhi value ho sakti hai), jo linear/identity deta hai. Classification ko aisi numbers chahiye jo classes mein probabilities ki tarah behave karein, jo softmax —

[0, 1]

mein bounded aur 1 mein summing — provide karta hai.

Hum weighted sum ko

W^{(ℓ)}

ki har row ke saath dot product ki tarah kyun likhte hain?

Har row ek neuron ke weights hold karti hai, toh ise

a^{(ℓ - 1)}

ke against multiply karna exactly

\sum_{j} w_{ij} a_{j}

hai — us single neuron ke liye evidence ka weighted aggregation. Row

i

⇒ neuron

i

Forward pass ke liye inputs normalise karna kyun matter karta hai jab math "kaam karta hai" dono taraf?

Altitude (

10000

) jaise feature ke saath Mach (

0.8

) ko dwarf karne par, weighted contributions ka ek column doosron ko swamp kar deta hai jab tak weights absurdly tiny na hon. Normalising features ko comparable scales par laata hai taaki ordinary size ke weights balanced rahein.

Edge cases

Woh boundaries jo topic quietly assume karti hai ki tum handle kar sakte ho.

Ek network ka forward pass output kya hoga jisme

L = 1

ho (input seedha output par, koi hidden layer nahi)?

Ek single affine-then-activation step:

y = σ^{(1)} (W^{(1)} x + b^{(1)})

. Bilkul valid — yeh sirf logistic/linear regression hai jab

σ^{(1)}

sigmoid/identity ho.

Agar input vector sab zeros ho toh network kya output karta hai?

Tab

z^{(1)} = b^{(1)}

(weight term vanish ho jaata hai), toh sirf biases hi layer 1 ko drive karte hain, aur pass normally proceed karta hai. Output generally nonzero hota hai un biases ki wajah se.

Agar ek ReLU layer ke SAARE pre-activations negative hon, toh us layer ka output kya hoga?

Zero vector. ReLU har entry ko 0 par clip kar deta hai, toh is input ke liye us layer se aage kuch bhi propagate nahi hota — ek genuine dead-layer moment, lekin sirf is particular input ke liye.

tanh ke liye, ek bahut bade pre-activation jaise

z_{4} = 4.54

ka kya hoga ek moderate wale ke comparison mein?

tanh large positive

z

ke liye

+ 1

ki taraf saturate ho jaata hai (isliye

tanh (4.54) \approx 0.9997

, lagbhag pinned), jabki moderate values zyaada gently map hote hain. Bahut large ya bahut small

z

dono

\pm 1

ke paas flatten ho jaate hain.

Do alag inputs same hidden activations

a^{(1)}

dete hain. Kya unhe same output

y

milna zaroori hai?

Haan.

a^{(1)}

se aage ka forward pass

a^{(1)}

ka ek deterministic function hai. Ek baar hidden activations match ho jaayein, toh downstream sab kuch match karta hai.

Agar

σ^{(L)}

identity hai, toh kya output layer "kuch nahi kar rahi"?

Nahi — yeh abhi bhi apne weights aur bias

W^{(L)} a^{(L - 1)} + b^{(L)}

apply kar rahi hai. Sirf final bending skip hoti hai; last hidden layer ka linear combination bilkul real kaam hai.

Kya ek hidden neuron ka bias akele use fire kara sakta hai jab usmein jaane wale saare weights zero hon?

Haan. Zero weights ke saath,

z_{i} = b_{i}

, toh ek positive bias ReLU se guzar ke ek constant positive output deta hai input se independent — ek constant-feature neuron. Unusual, lekin math allow karta hai.

Recall One-sentence self-test

Bina dekhe, forward pass ki har layer par execute hone wala do-part rule batao. ::: Pre-activation compute karo $z^{(ℓ)} = W^{(ℓ)} a^{(ℓ - 1)} + b^{(ℓ)}$ , phir activation $a^{(ℓ)} = σ^{(ℓ)} (z^{(ℓ)})$ .