Convolutional neural networks — convolution operation, pooling

5.6.11 · Coding › Machine Learning (Aerospace Applications)

Intuition Badi wali idea

CNN ek aisa neural network hai jo weights ko space mein share karta hai. Har pixel ke liye alag weight seekhne ki jagah (jo wasteful hai aur structure ko ignore karta hai), CNN ek chota filter image ke upar slide karta hai aur same weights ko har jagah reuse karta hai. Isse ek saath do kaam hote hain:

Translation equivariance: wing ka edge, wing ka edge hi hota hai — chahe woh top-left mein ho ya bottom-right mein.
Parameter reduction: ek $3 \times 3$ filter mein sirf 9 weights hote hain, na ki har pixel ke liye ek.

Aerospace mein: composite panel images mein cracks detect karna, runways ki satellite imagery classify karna, ya CFD pressure fields se features extract karna — CNNs us local spatial structure ko exploit karte hain jo ek fully-connected net barbaad kar deta hai.

1. The Convolution Operation

YEH HAI KYA

YEH KAAM KYUN KARTA HAI — scratch se derive karo

Ek fully-connected layer se shuru karo: har output unit $y_{p} = \sum_{q} W_{pq} x_{q} + b_{p}$ . Ek $100 \times 100$ image jo ek $100 \times 100$ hidden layer ko feed kar rahi hai, usmein $1 0^{8}$ weights hote hain. Do problems hain:

Koi spatial prior nahi — pixel $(0, 0)$ aur $(99, 99)$ ko independent weights milte hain, isliye net yeh "jaan" nahi sakta ki nearby pixels zyada matter karte hain.
Koi reuse nahi — ek location par seekha gaya feature har jagah dobara seekhna padega.

$W$ par do constraints lagao:

Locality: $W_{pq} = 0$ unless $q$ , $p$ ke ek chhote $k \times k$ neighbourhood mein ho.
Weight sharing: same $k^{2}$ weights har output $p$ ke liye use hote hain.

Inn constraints ko $y_{p} = \sum_{q} W_{pq} x_{q}$ mein substitute karne par giant matrix ek single tiny kernel $K$ mein collapse ho jaata hai jo image ke upar slide karta hai — aur convolution formula upar wala nikal aata hai. Convolution sirf ek constrained fully-connected layer hai. Yahi hai WHY.

OUTPUT SIZE KYUN CHHOTA HOTA HAI

Figure — Convolutional neural networks — convolution operation, pooling

Channels

Real images mein $C$ channels hote hain (RGB, ya stacked physics fields). Tab ek filter $k \times k \times C$ hota hai; tum channels ke upar bhi sum karte ho. $F$ filters ke saath output mein $F$ channels hote hain. Parameters $= F (k^{2} C + 1)$ .

2. Pooling

3. Common Mistakes (Steel-manned)

4. Flashcards

Kaun se do constraints ek fully-connected layer ko convolution mein badal dete hain?

Locality (weights sirf ek chhote neighbourhood mein nonzero hain) aur weight sharing (same weights har position par).

Conv/pool layer ke liye output-size formula

W_{o u t} = ⌊(W_{in} - k + 2 p) / s ⌋ + 1

Output-size formula mein

+ 1

kyun aata hai?

Yeh kernel ki starting position 0 ko, uske baad ke strided positions ke alaawa, count karta hai.

Conv layer mein kitne learnable parameters hote hain (F filters, k×k, C channels)?

F (k^{2} C + 1)

, jahan

+ 1

har filter ka bias hai; image size se independent.

Kya pooling mein learnable parameters hote hain?

Nahi — yeh ek fixed max/average function hai.

Pooling use karne ke do reasons batao.

Local translation invariance aur dimensionality reduction (receptive field bhi bada hota hai).

CNN mein kernel flipping kyun matter nahi karta?

Kernel weights seekhe jaate hain, isliye network simply flipped values seekh leta hai.

Weight sharing CNN ko kya property deta hai?

Translation equivariance — same feature position ki parwah kiye bina detect hota hai.

Output size compute karo: input 32, kernel 5, pad 2, stride 1.

(32 - 5 + 4) /1 + 1 = 32

(padding size preserve karta hai).

Max-pool vs average-pool intuition?

Max sabse strong "feature present hai?" signal rakhta hai; average smooth/overall magnitude retain karta hai.

Recall Feynman: 12-saal ke bachche ko explain karo

Socho tumhare paas ek chhoti stamp hai jis par ek pattern bana hai, aur stickers ki ek badi sheet hai. Tum same stamp ko poori sheet par dabate ho aur mark karte ho jahan bhi pattern match karta hai — yeh stamping convolution hai, aur ek hi stamp ko har jagah use karne ka matlab hai ki tumhe har spot ke liye alag rule yaad nahi karna. Phir tum marked sheet ko chhote $2 \times 2$ squares mein dekhte ho aur, har square mein, sirf sabse loud mark rakhte ho — yahi max-pooling hai. Ab sheet chhoti hai aur tumhe abhi bhi yaad hai ki kahan kaisi cheezein hain, chahe woh thodi si hil bhi gayi hon. Bahut sari stamps aur shrinks stack karo, aur computer wing mein cracks ya sky se runways recognize karna seekh jaata hai.

Connections

Fully-connected neural networks — conv iska ek constrained version hai.
Backpropagation — gradients conv (shared-weight sum) aur pooling (max argmax tak gradient route karta hai) ke through flow karte hain.
Feature maps and receptive fields
Padding and stride
Image classification for aerospace inspection — crack/defect detection.
Translation equivariance vs invariance