Pooling layers (max, average)

Overview

Pooling layers reduce the spatial dimensions (height and width) of feature maps while retaining the most important information. They act as a downsampling operation that makes the network more computationally efficient and robust to small translations in the input.

Think of pooling as summarizing neighborhoods: "What's the most important feature in this 2×2 region?"

Max Pooling: Derivation from First Principles

The Core Idea

For a region of the input feature map, take the maximum value. This preserves the strongest activation.

Y_{i,j} = \max_{m=0}^{p-1} \max_{n=0}^{p-1} X_{i \cdot s + m, \; j \cdot s + n}

where:

$(i, j)$ indexes the output feature map
$(m, n)$ indexes within the pooling window
$s$ is the stride (typically $s = p$ for non-overlapping pooling)

Why max? In CNNs, activations represent feature presence. A high value means "this feature is strongly present here." Taking the max preserves the strongest signal in the region.

Output Dimension Calculation

Starting from the input dimension formula:

H_{out} = \left\lfloor \frac{H_{in} - p}{s} \right\rfloor + 1

W_{out} = \left\lfloor \frac{W_{in} - p}{s} \right\rfloor + 1

Derivation:

The first pooling window starts at position 0
Each subsequent window starts at position $s, 2s, 3s, \ldots$
The last valid window must fit entirely: starting position $\leq H_{in} - p$
Number of valid positions: $\lfloor (H_{in} - p)/s \rfloor + 1$

For the common case where $s = p$ (non-overlapping):

H_{out} = \frac{H_{in}}{p}, \quad W_{out} = \frac{W_{in}}{p}

Input:
[1  3  2  4]
[5  6  7  8]
[9  10 1  2]
[3  4  5  6]

Operation: 2×2 max pooling, stride 2

Step 1: Top-left 2×2 window

[1  3]
[5  6]  →  max = 6

Why this step? We scan the first 2×2 region and pick the strongest activation.

Step 2: Top-right 2×2 window

[2  4]
[7  8]  →  max = 8

Step 3: Bottom-left 2×2 window

[9  10]
[3  4]  →  max = 10

Step 4: Bottom-right 2×2 window

[1  2]
[5  6]  →  max = 6

Output: 2×2 feature map

[6   8]
[10  6]

Verification: Input was 4×4, output is 2×2 → dimension reduced by factor of 2.

Average Pooling: Derivation from First Principles

The Core Idea

Instead of taking the maximum, compute the average of all values in the pooling window.

Why average? This preserves aggregate information about a region rather than just the strongest signal. Useful when you want to capture overall feature presence, not just peak activations.

The output dimensions are calculated identically to max pooling.

[1  3  2  4]
[5  6  7  8]
[9  10 1  2]
[3  4  5  6]

Operation: 2×2 average pooling, stride 2

Step 1: Top-left

[1  3]
[5  6]  →  (1+3+5+6)/4 = 15/4 = 3.75

Why divide by 4? The window has 4 elements; we want the mean.

Step 2: Top-right

[2  4]
[7  8]  →  (2+4+7+8)/4 = 21/4 = 5.25

Step 3: Bottom-left

[9  10]
[3  4]  →  (9+10+3+4)/4 = 26/4 = 6.5

Step 4: Bottom-right

[1  2]
[5  6]  →  (1+2+5+6)/4 = 14/4 = 3.5

Output:

[3.75  5.25]
[6.5   3.5]

Notice how the values are smoother compared to max pooling.

Max vs Average: When to Use Which?

Criterion	Max Pooling	Average Pooling
Feature detection	Preserves strongest signals	Preserves average presence
Use case	Object detection, classification	Texture analysis, background features
Typical placement	Hidden layers (conv → max → conv)	Final layers before fully connected
Gradient flow	Sparse (only to max element)	Dense (to all elements)
Robustness	More robust to noise spikes	Affected by all values equally

Intuition:

Max pooling = "Is this feature present anywhere in this region?" → Binary-like detection
Average pooling = "How much of this feature is in this region on average?" → Continuous measure

Backpropagation Through Pooling

Max Pooling Gradient

The gradient only flows back to the position that had the maximum value during the forward pass.

\frac{\partial \mathcal{L}}{\partial X_{i,j}} = \begin{cases} \frac{\partial \mathcal{L}}{\partial Y_{k,l}} & \text{if } X_{i,j} = \max \text{ in window for } Y_{k,l} \\ 0 & \text{otherwise} \end{cases}

Why? During forward pass, only the max value affected the output. By the chain rule, gradients flow only through the path that was actually used.

Worked example Max Pooling Backprop

Forward pass:

Input:          Pooling output:
[1  3  2  4]    [6   8]
[5  6  7  8]    [10  6]
[9  10 1  2]
[3  4  5  6]

Backward pass: Suppose gradient from next layer is:

dL/dY:
[0.5  1.0]
[0.3  0.7]

The gradient flows to:

dL/dX:
[0    0    0    0  ]
[0    0.5  0    1.0]
[0    0.3  0    0  ]
[0    0    0    0.7]

Why these positions? Position (1,1) had value 6 (max of top-left), so it gets gradient 0.5. Position (1,3) had value 8 (max of top-right), so it gets 1.0. And so on.

Average Pooling Gradient

The gradient is distributed equally to all positions in the pooling window:

\frac{\partial \mathcal{L}}{\partial X_{i,j}} = \frac{1}{p^2} \frac{\partial \mathcal{L}}{\partial Y_{k,l}}

for all positions $(i,j)$ in the window that produced $Y_{k,l}$ .

Common Mistakes

Why it feels right: Other layers like convolutions have weights, so pooling should too.

The fix: Pooling is a fixed operation (max or mean). It has zero learnable parameters. The "intelligence" comes from the learned feature maps it operates on, not the pooling itself.

Steel-man: It's reasonable to think downsampling requires learned weights. But pooling's strength is its simplicity—it reduces overfitting precisely because it doesn't add parameters.

Why it feels right: Non-overlapping windows seem cleaner and are most common (e.g., 2×2 pool with stride 2).

The fix: You can use stride < pool size for overlapping pooling. For example, 3×3 pool with stride 2 creates overlaps. This can improve accuracy but increases computation.

Formula for overlapping: If $s < p$ , regions overlap by $p - s$ pixels.

Why it feels right: Max pooling dominates in architectures like VGG, ResNet early layers.

The fix: Average pooling often works better in:

Global pooling (replacing fully connected layers): GAP preserves spatial information better
Shallow features: Where you want smooth, continuous representations
Segmentation tasks: Where spatial information is critical

Modern architectures like MobileNet and EfficientNet use average pooling extensively.

Why it feels right: This is the most common configuration.

The fix: Output size depends on input size, pool size, and stride via the formula:

H_{out} = \left\lfloor \frac{H_{in} - p}{s} \right\rfloor + 1

Example: 7×7 input, 3×3 pool, stride 2 → $\lfloor (7-3)/2 \rfloor + 1 = 3$ → 3×3 output.

Advanced Concepts

Global Pooling

Global Average Pooling (GAP) and Global Max Pooling (GMP) reduce each entire feature map to a single value:

\text{GAP}: \quad y = \frac{1}{H \times W} \sum_{i=1}^{H} \sum_{j=1}^{W} X_{i,j}

\text{GMP}: \quad y = \max_{i,j} X_{i,j}

Why? Replaces fully connected layers, reducing parameters dramatically. Used in Network-in-Network, ResNet, Inception.

Stochastic Pooling

During training, sample from the pooling window based on a probability distribution (values normalized). During testing, use weighted average. This adds regularization.

Recall Explain to a 12-Year-Old

Imagine you're looking at a HUGE poster of a soccer game, but you need to describe it quickly on a small notecard. You can't write everything!

Max pooling is like picking the most exciting moment from each section: "Top-left? The goalie made a HUGE jump—score 10!" You write 10. "Top-right? Best player scored—score 8!" You write 8. You're keeping the BEST stuff from each part.

Average pooling is like giving each section an overall rating: "Top-left had a jump (10), some running (5), some standing (2)—average is about 6." You're keeping the GENERAL feel, not just the peaks.

Both make the poster smaller, but max pooling keeps drama, average pooling keeps the overall vibe. In computer vision, "drama" = strong features like edges and corners!

Picture: A mountain range (max pooling) vs. rolling hills (average pooling).

Connections

Convolutional Layers: Pooling always follows conv layers to downsample feature maps
Receptive Field: Pooling increases effective receptive field of deeper layers
Translation Invariance: Pooling is a key mechanism for achieving this property
Global Average Pooling: Replaces fully connected layers in modern architectures
Stride and Padding: Pooling uses stride to control overlap; padding less common
ResNet Architecture: Uses max pooling early, average pooling before final classifier
VGG Network: Uses 2×2 max pooling after every 2-3 conv layers
Feature Maps: Pooling operates on these to reduce spatial dimensions

Summary

Pooling layers are fixed downsampling operations that:

Reduce spatial dimensions by factor of pool size (typically 2)
Have zero learnable parameters
Increase translation invariance and computational efficiency
Come in two main types: max (preserves strongest signals) and average (preserves aggregate information)
Propagate gradients differently: max (sparse to max element), average (distributed to all)

The choice between max and average depends on your task: max for feature detection (classification), average for holistic representations (often in final layers).

#flashcards/ai-ml

What is the purpose of pooling layers in CNNs?

Pooling layers downsample spatial dimensions (H×W) of feature maps while retaining important information, reducing computation, controlling overfitting, and creating translation invariance.

How does max pooling operation work mathematically?

For each pooling window, max pooling selects the maximum value:

Y_{i,j} = \max_{m,n} X_{i \cdot s + m, j \cdot s + n}

where the max is taken over the window of size p×p.

What is the output dimension formula for pooling?

H_{out} = \lfloor (H_{in} - p)/s \rfloor + 1

and

W_{out} = \lfloor (W_{in} - p)/s \rfloor + 1

, where p is pool size and s is stride. For s=p (non-overlapping), dimensions reduce by factor of p.

How does average pooling differ from max pooling?

Average pooling computes the mean of all values in the window:

Y_{i,j} = \frac{1}{p^2} \sum_{m,n} X_{i \cdot s + m, j \cdot s + n}

, preserving aggregate information rather than just peak activations.

Do pooling layers have learnable parameters?

No, pooling layers have ZERO learnable parameters. They are fixed operations (max or mean) that don't require training.

How do gradients flow in max pooling backpropagation?

Gradients flow ONLY to the position that had the maximum value during forward pass. All other positions receive zero gradient.

How do gradients flow in average pooling backpropagation?

Gradients are distributed EQUALLY to all positions in the pooling window, each receiving (1/p²) × gradient from output.

When should you use max pooling vs average pooling?

Max pooling for feature detection and classification (preserves strongest signals). Average pooling for texture analysis and often in final layers before fully connected (preserves aggregate information).

What is Global Average Pooling (GAP)?

GAP reduces each entire feature map to a single value by averaging all spatial positions:

y = \frac{1}{H \times W} \sum_{i,j} X_{i,j}

. Replaces fully connected layers in modern architectures.

Can pooling windows overlap?

Yes, when stride < pool size. For example, 3×3 pool with stride 2 creates overlapping regions, which can improve accuracy at the cost of more computation.

What is the typical pooling configuration in CNNs?

2×2 pooling with stride 2 (non-overlapping), which reduces spatial dimensions by exactly half in each direction.

Why does pooling create translation invariance?

By summarizing local regions, pooling makes the network less sensitive to exact spatial positions. A feature shifted by a few pixels still produces similar output after pooling.

Concept Map

Hinglish (regional understanding)

Intuition Hinglish mein samjho

Hinglish (regional understanding)

Intuition Hinglish mein samjho

Chalo isko simple tarike se samajhte hain. Jab CNN ek image process karti hai, to har layer feature maps banati hai jo kaafi bade hote hain aur unme bahut saara data hota hai. Pooling ka kaam hai in feature maps ko chota karna, yaani spatial dimensions (height aur width) ko kam karna, lekin important information ko bachaye rakhte hue. Isko aise socho jaise ek 2x2 chota sa area (neighborhood) le rahe ho aur us poore area ko ek single number se represent kar rahe ho. Max pooling me tum us area ka sabse bada value uthate ho (sabse strong feature), aur average pooling me sabhi values ka average nikaalte ho. Simple, na?

Ab sawaal ye hai ki ye matter kyun karta hai? Sabse pehli baat, translation invariance milti hai — matlab agar image me koi cheez thodi si left ya right shift ho jaye, tab bhi network use pehchan lega, kyunki tum poore neighborhood ka summary le rahe ho, exact pixel position nahi. Doosra, computation kam ho jaata hai — jab feature map chota hoga to deeper layers me kaam karna fast aur efficient ho jaata hai. Teesra, ye overfitting ko control karta hai, kyunki network exact positions ko memorize nahi kar paata, sirf general patterns seekhta hai.

Dimension calculation bhi yaad rakhna zaroori hai: agar stride $s$ aur window size $p$ same ho (jaise 2x2 with stride 2), to output dimension bilkul aadha ho jaata hai — 4x4 se 2x2. Formula hai $H_{out} = \lfloor (H_{in} - p)/s \rfloor + 1$ . Practical me max pooling zyada common hai kyunki wo sabse strong activations ko preserve karta hai, jo usually most important features hote hain. To beta, pooling ko ek "smart summarizer" samjho jo network ko chota, fast aur robust banata hai bina zyada important information kho diye.

Overview

Max Pooling: Derivation from First Principles

The Core Idea

Output Dimension Calculation

Average Pooling: Derivation from First Principles

The Core Idea

Max vs Average: When to Use Which?

Backpropagation Through Pooling

Max Pooling Gradient

Average Pooling Gradient

Common Mistakes

Advanced Concepts

Global Pooling

Stochastic Pooling

Connections

Summary

Concept Map

Hinglish (regional understanding)

Hinglish (regional understanding)

Go deeper — visual, from zero

Test yourself — Convolutional Neural Networks

Connections