Pooling layers (max, average) — Hinglish notes

3.4.3 · AI-ML › Convolutional Neural Networks

Overview

Pooling layers feature maps ki spatial dimensions (height aur width) ko reduce karte hain, jabki sabse important information retain hoti hai. Ye ek downsampling operation ki tarah kaam karte hain jo network ko computationally zyada efficient banata hai aur input mein chhoti translations ke liye robust banata hai.

Pooling ko neighborhoods ko summarize karna samjho: "Is 2×2 region mein sabse important feature kya hai?"

Max Pooling: First Principles Se Derivation

Core Idea

Input feature map ke ek region ke liye, maximum value lo. Yeh sabse strong activation preserve karta hai.

Y_{i, j} = m = 0 max p - 1 n = 0 max p - 1 X_{i \cdot s + m, j \cdot s + n}

jahan:

$(i, j)$ output feature map ko index karta hai
$(m, n)$ pooling window ke andar index karta hai
$s$ stride hai (typically $s = p$ non-overlapping pooling ke liye)

Max kyun? CNNs mein, activations feature presence represent karte hain. Ek high value matlab "yeh feature yahan strongly present hai." Max lene se region mein sabse strong signal preserve hota hai.

Output Dimension Calculation

Input dimension formula se shuru karke:

H_{o u t} = ⌊ \frac{H _{in} - p}{s} ⌋ + 1

W_{o u t} = ⌊ \frac{W _{in} - p}{s} ⌋ + 1

Derivation:

Pehli pooling window position 0 se start hoti hai
Har baad wali window position $s, 2 s, 3 s, \dots$ se start hoti hai
Aakhri valid window poori fit honi chahiye: starting position $\leq H_{in} - p$
Valid positions ki number: $⌊(H_{in} - p) / s ⌋ + 1$

Common case mein jahan $s = p$ (non-overlapping):

H_{o u t} = \frac{H _{in}}{p}, W_{o u t} = \frac{W _{in}}{p}

Input:
[1  3  2  4]
[5  6  7  8]
[9  10 1  2]
[3  4  5  6]

Operation: 2×2 max pooling, stride 2

Step 1: Top-left 2×2 window

[1  3]
[5  6]  →  max = 6

Yeh step kyun? Hum pehle 2×2 region ko scan karte hain aur sabse strong activation pick karte hain.

Step 2: Top-right 2×2 window

[2  4]
[7  8]  →  max = 8

Step 3: Bottom-left 2×2 window

[9  10]
[3  4]  →  max = 10

Step 4: Bottom-right 2×2 window

[1  2]
[5  6]  →  max = 6

Output: 2×2 feature map

[6   8]
[10  6]

Verification: Input 4×4 tha, output 2×2 hai → dimension 2 ke factor se reduce hua.

Average Pooling: First Principles Se Derivation

Core Idea

Maximum lene ki jagah, pooling window ke sabhi values ka average compute karo.

Average kyun? Yeh ek region ke baare mein aggregate information preserve karta hai, na ki sirf sabse strong signal. Tab useful hota hai jab tum overall feature presence capture karna chahte ho, na ki sirf peak activations.

Output dimensions wahi formula se calculate hote hain jo max pooling mein use hoti hai.

[1  3  2  4]
[5  6  7  8]
[9  10 1  2]
[3  4  5  6]

Operation: 2×2 average pooling, stride 2

Step 1: Top-left

[1  3]
[5  6]  →  (1+3+5+6)/4 = 15/4 = 3.75

4 se divide kyun? Window mein 4 elements hain; hum mean chahte hain.

Step 2: Top-right

[2  4]
[7  8]  →  (2+4+7+8)/4 = 21/4 = 5.25

Step 3: Bottom-left

[9  10]
[3  4]  →  (9+10+3+4)/4 = 26/4 = 6.5

Step 4: Bottom-right

[1  2]
[5  6]  →  (1+2+5+6)/4 = 14/4 = 3.5

Output:

[3.75  5.25]
[6.5   3.5]

Dhyan do ki values max pooling ke comparison mein kitni smoother hain.

Max vs Average: Kab Kaunsa Use Karein?

Criterion	Max Pooling	Average Pooling
Feature detection	Sabse strong signals preserve karta hai	Average presence preserve karta hai
Use case	Object detection, classification	Texture analysis, background features
Typical placement	Hidden layers (conv → max → conv)	Fully connected se pehle final layers mein
Gradient flow	Sparse (sirf max element tak)	Dense (sabhi elements tak)
Robustness	Noise spikes ke liye zyada robust	Sabhi values se equally affected

Intuition:

Max pooling = "Kya yeh feature is region mein kahin bhi present hai?" → Binary-jaisi detection
Average pooling = "Is region mein average par yeh feature kitna hai?" → Continuous measure

Pooling Se Backpropagation

Max Pooling Gradient

Gradient sirf us position tak wapas flow karta hai jahan forward pass mein maximum value thi.

\frac{\partial L}{\partial X _{i, j}} = {\frac{\partial L}{\partial Y _{k, l}} 0 if X_{i, j} = max in window for Y_{k, l} otherwise

Kyun? Forward pass ke dauran, sirf max value ne output ko affect kiya. Chain rule se, gradients sirf us path se flow karte hain jo actually use hua tha.

Worked example Max Pooling Backprop

Forward pass:

Input:          Pooling output:
[1  3  2  4]    [6   8]
[5  6  7  8]    [10  6]
[9  10 1  2]
[3  4  5  6]

Backward pass: Suppose next layer se gradient yeh hai:

dL/dY:
[0.5  1.0]
[0.3  0.7]

Gradient yahan flow karta hai:

dL/dX:
[0    0    0    0  ]
[0    0.5  0    1.0]
[0    0.3  0    0  ]
[0    0    0    0.7]

Yeh positions kyun? Position (1,1) ki value 6 thi (top-left ka max), isliye use gradient 0.5 milta hai. Position (1,3) ki value 8 thi (top-right ka max), isliye use 1.0 milta hai. Aur aise hi aage.

Average Pooling Gradient

Gradient pooling window ke sabhi positions mein equally distribute hota hai:

\frac{\partial L}{\partial X _{i, j}} = \frac{1}{p ^{2}} \frac{\partial L}{\partial Y _{k, l}}

un sabhi positions $(i, j)$ ke liye jo window mein hain aur jinhone $Y_{k, l}$ produce kiya.

Common Mistakes

Kyun sahi lagta hai: Doosre layers jaise convolutions mein weights hote hain, toh pooling mein bhi hone chahiye.

Fix: Pooling ek fixed operation hai (max ya mean). Iske zero learnable parameters hain. "Intelligence" seekhe hue feature maps se aati hai, pooling se nahi.

Steel-man: Yeh sochna reasonable hai ki downsampling ke liye learned weights chahiye. Lekin pooling ki taakat uski simplicity mein hai—yeh overfitting reduce karta hai kyunki yeh parameters add nahi karta.

Kyun sahi lagta hai: Non-overlapping windows zyada clean lagte hain aur sabse common hain (jaise 2×2 pool with stride 2).

Fix: Tum overlapping pooling ke liye stride < pool size use kar sakte ho. For example, 3×3 pool with stride 2 overlaps create karta hai. Yeh accuracy improve kar sakta hai lekin computation badhata hai.

Overlapping ke liye formula: Agar $s < p$ , toh regions $p - s$ pixels se overlap karte hain.

Kyun sahi lagta hai: Max pooling VGG, ResNet early layers jaise architectures mein dominate karta hai.

Fix: Average pooling aksar inme better kaam karta hai:

Global pooling (fully connected layers replace karna): GAP spatial information better preserve karta hai
Shallow features: Jahan tum smooth, continuous representations chahte ho
Segmentation tasks: Jahan spatial information critical hai

Modern architectures jaise MobileNet aur EfficientNet extensively average pooling use karte hain.

Kyun sahi lagta hai: Yeh sabse common configuration hai.

Fix: Output size, input size, pool size, aur stride par depend karta hai is formula se:

H_{o u t} = ⌊ \frac{H _{in} - p}{s} ⌋ + 1

Example: 7×7 input, 3×3 pool, stride 2 → $⌊(7 - 3) /2 ⌋ + 1 = 3$ → 3×3 output.

Advanced Concepts

Global Pooling

Global Average Pooling (GAP) aur Global Max Pooling (GMP) har poore feature map ko ek single value mein reduce kar dete hain:

GAP : y = \frac{1}{H \times W} i = 1 \sum H j = 1 \sum W X_{i, j}

GMP : y = i, j max X_{i, j}

Kyun? Fully connected layers replace karta hai, parameters dramatically reduce karta hai. Network-in-Network, ResNet, Inception mein use hota hai.

Stochastic Pooling

Training ke dauran, pooling window se ek probability distribution ke basis par sample karo (values normalize ki jaati hain). Testing ke dauran, weighted average use karo. Yeh regularization add karta hai.

Recall Ek 12-Saal Ke Bachche Ko Explain Karo

Socho tum ek BAHUT BADA soccer game ka poster dekh rahe ho, lekin ek chhote notecard par jaldi describe karna hai. Sab kuch likhna possible nahi!

Max pooling aise hai jaise har section se sabse exciting moment pick karna: "Top-left? Goalkeeper ne BAHUT badi jump ki—score 10!" Tum 10 likhte ho. "Top-right? Best player ne score kiya—score 8!" Tum 8 likhte ho. Tum har part se BEST cheez rakh rahe ho.

Average pooling aise hai jaise har section ko overall rating dena: "Top-left mein ek jump thi (10), kuch running (5), kuch khade rehna (2)—average lagbhag 6 hai." Tum GENERAL feel rakh rahe ho, sirf peaks nahi.

Dono poster ko chhota karte hain, lekin max pooling drama rakhta hai, average pooling overall vibe rakhta hai. Computer vision mein, "drama" = strong features jaise edges aur corners!

Picture karo: Ek mountain range (max pooling) vs. rolling hills (average pooling).

Connections

Convolutional Layers: Pooling hamesha feature maps ko downsample karne ke liye conv layers ke baad aata hai
Receptive Field: Pooling deeper layers ka effective receptive field badhata hai
Translation Invariance: Pooling is property achieve karne ka ek key mechanism hai
Global Average Pooling: Modern architectures mein fully connected layers replace karta hai
Stride and Padding: Pooling overlap control karne ke liye stride use karta hai; padding kam common hai
ResNet Architecture: Early mein max pooling use karta hai, final classifier se pehle average pooling
VGG Network: Har 2-3 conv layers ke baad 2×2 max pooling use karta hai
Feature Maps: Pooling spatial dimensions reduce karne ke liye inhi par operate karta hai

Summary

Pooling layers fixed downsampling operations hain jo:

Pool size ke factor se spatial dimensions reduce karte hain (typically 2)
Zero learnable parameters rakhte hain
Translation invariance aur computational efficiency badhate hain
Do main types mein aate hain: max (sabse strong signals preserve karta hai) aur average (aggregate information preserve karta hai)
Gradients alag tarike se propagate karte hain: max (max element tak sparse), average (sabhi mein distributed)

Max aur average ke beech choice tumhare task par depend karti hai: feature detection ke liye max (classification), holistic representations ke liye average (aksar final layers mein).

#flashcards/ai-ml

CNNs mein pooling layers ka purpose kya hai?

Pooling layers feature maps ki spatial dimensions (H×W) ko downsample karti hain, important information retain karti hain, computation reduce karti hain, overfitting control karti hain, aur translation invariance create karti hain.

Max pooling operation mathematically kaise kaam karta hai?

Har pooling window ke liye, max pooling maximum value select karta hai:

Y_{i, j} = max_{m, n} X_{i \cdot s + m, j \cdot s + n}

jahan max p×p size ki window par liya jaata hai.

Pooling ke liye output dimension formula kya hai?

H_{o u t} = ⌊(H_{in} - p) / s ⌋ + 1

aur

W_{o u t} = ⌊(W_{in} - p) / s ⌋ + 1

, jahan p pool size hai aur s stride hai. s=p (non-overlapping) ke liye, dimensions p ke factor se reduce hoti hain.

Average pooling, max pooling se kaise alag hai?

Average pooling window ke sabhi values ka mean compute karta hai:

Y_{i, j} = \frac{1}{p ^{2}} \sum_{m, n} X_{i \cdot s + m, j \cdot s + n}

, sirf peak activations ki jagah aggregate information preserve karta hai.

Kya pooling layers mein learnable parameters hote hain?

Nahi, pooling layers mein ZERO learnable parameters hote hain. Ye fixed operations hain (max ya mean) jo training ki zaroorat nahi rakhte.

Max pooling backpropagation mein gradients kaise flow karte hain?

Gradients SIRF us position tak flow karte hain jahan forward pass mein maximum value thi. Baaki sabhi positions ko zero gradient milta hai.

Average pooling backpropagation mein gradients kaise flow karte hain?

Gradients pooling window ke sabhi positions mein EQUALLY distribute hote hain, har ek ko (1/p²) × output ka gradient milta hai.

Max pooling vs average pooling kab use karna chahiye?

Feature detection aur classification ke liye max pooling (sabse strong signals preserve karta hai). Texture analysis aur aksar final layers mein fully connected se pehle average pooling (aggregate information preserve karta hai).

Global Average Pooling (GAP) kya hai?

GAP har poore feature map ko ek single value mein reduce karta hai sabhi spatial positions ko average karke:

y = \frac{1}{H \times W} \sum_{i, j} X_{i, j}

. Modern architectures mein fully connected layers replace karta hai.

Kya pooling windows overlap kar sakti hain?

Haan, jab stride < pool size ho. For example, 3×3 pool with stride 2 overlapping regions create karta hai, jo zyada computation ki cost par accuracy improve kar sakta hai.

CNNs mein typical pooling configuration kya hai?

2×2 pooling with stride 2 (non-overlapping), jo spatial dimensions ko har direction mein exactly half reduce karta hai.

Pooling translation invariance kyun create karta hai?

Local regions ko summarize karke, pooling network ko exact spatial positions ke liye kam sensitive banata hai. Kuch pixels se shift hua feature pooling ke baad bhi similar output produce karta hai.