Pooling layers (max, average): Exercises — AI-ML deep dive

This page is a self-test. Each problem builds on the parent note Pooling Layers. Solutions are hidden inside collapsible callouts — try first, then reveal.

Before we start, one reminder of the only two formulas you need for sizing. If a feature map has height $H_{in}$ , we slide a window of side $p$ (the "pool size") and jump $s$ pixels each time (the "stride"). The number of window positions that fit is:

The two operations themselves, on one window of numbers:

Max pooling keeps the single largest number.
Average pooling adds all $p^{2}$ numbers and divides by $p^{2}$ (the mean).

Prerequisites if any symbol feels new: Feature Maps, Stride and Padding, Receptive Field.

Level 1 — Recognition

Exercise 1.1

An input feature map is $6 \times 6$ . You apply $2 \times 2$ max pooling with stride $2$ . What is the output height and width?

Recall Solution

Plug into the formula with $H_{in} = 6$ , $p = 2$ , $s = 2$ : $H_{o u t} = ⌊ \frac{6 - 2}{2} ⌋ + 1 = ⌊ 2 ⌋ + 1 = 3$ By symmetry $W_{o u t} = 3$ too. Output: $3 \times 3$ . Sanity check: non-overlapping $2 \times 2$ pooling halves each side, and $6/2 = 3$ . ✓

Exercise 1.2

Look at the single window below. State the max-pooling result and the average-pooling result.

[2  9]
[4  1]

Recall Solution

Max: the largest of ${2, 9, 4, 1}$ is $9$ . Average: $\frac{2 + 9 + 4 + 1}{4} = \frac{16}{4} = 4$ . Notice max ( $9$ ) is pulled toward the peak; average ( $4$ ) sits in the middle of all four.

Exercise 1.3

True or false: a pooling layer has learnable weights that get updated during training.

Recall Solution

False. Both max and average pooling are fixed operations — no weights, no biases, nothing to learn. Contrast this with Convolutional Layers, whose filters do have learnable weights.

Level 2 — Application

Exercise 2.1

Apply $2 \times 2$ max pooling, stride $2$ , to this $4 \times 4$ map. Give the full $2 \times 2$ output.

[3  8  1  0]
[2  5  9  4]
[7  6  2  2]
[1  0  3  8]

Recall Solution

Four non-overlapping windows (see the four coloured tiles in the figure):

Top-left ${3, 8, 2, 5}$ → max $= 8$
Top-right ${1, 0, 9, 4}$ → max $= 9$
Bottom-left ${7, 6, 1, 0}$ → max $= 7$
Bottom-right ${2, 2, 3, 8}$ → max $= 8$

$Y = [8798]$

Exercise 2.2

An input is $7 \times 7$ . Apply $3 \times 3$ pooling with stride $2$ . What is the output size? Does any input row get ignored?

Recall Solution

$H_{o u t} = ⌊ \frac{7 - 3}{2} ⌋ + 1 = ⌊ 2 ⌋ + 1 = 3.$ Output: $3 \times 3$ . Window start positions along one axis are $0, 2, 4$ . The last window covers rows $4, 5, 6$ — so row 6 is the final row used and nothing is ignored here. (If the input were $8$ : $⌊(8 - 3) /2 ⌋ + 1 = 3$ , starts $0, 2, 4$ , last window covers rows $4, 5, 6$ and row 7 is dropped by the floor.)

Exercise 2.3

Apply $2 \times 2$ average pooling, stride $2$ , to the same map as Exercise 2.1. Give the output.

Recall Solution

Top-left $(3 + 8 + 2 + 5) /4 = 18/4 = 4.5$
Top-right $(1 + 0 + 9 + 4) /4 = 14/4 = 3.5$
Bottom-left $(7 + 6 + 1 + 0) /4 = 14/4 = 3.5$
Bottom-right $(2 + 2 + 3 + 8) /4 = 15/4 = 3.75$

$Y = [4.5 3.5 3.5 3.75]$ Notice every output is smaller and smoother than the max version — averaging drags peaks down toward the crowd.

Level 3 — Analysis

Exercise 3.1

Forward pass of Exercise 2.1 gave output $[8798]$ . The next layer sends back the gradient

dL/dY:
[0.2  0.5]
[0.1  0.4]

Route this gradient back through the max pooling to fill in the $4 \times 4$ dL/dX.

Recall Solution

In max pooling, only the winning position in each window carried information forward, so by the chain rule only it receives gradient; every other cell gets $0$ .

Top-left winner was $8$ at row 0, col 1 → gets $0.2$
Top-right winner was $9$ at row 1, col 2 → gets $0.5$
Bottom-left winner was $7$ at row 2, col 0 → gets $0.1$
Bottom-right winner was $8$ at row 3, col 3 → gets $0.4$

$\frac{\partial L}{\partial X} = 00 0.1 0 0.2 000 0 0.5 00 000 0.4$ Sum of all entries $= 0.2 + 0.5 + 0.1 + 0.4 = 1.2$ , which equals the sum of dL/dY — no gradient is created or destroyed, just re-routed.

Exercise 3.2

Route the same dL/dY back through average pooling ( $p = 2$ ) instead. What does the top-left window's dL/dX look like?

Recall Solution

Average pooling used all four inputs equally (each contributed $1/ p^{2} = 1/4$ to the output). So the gradient is split evenly among the four: $\frac{1}{4} \times 0.2 = 0.05 for each of the four top-left cells.$ $[0.05 0.05 0.05 0.05]$ Compare to max pooling, where a single cell got the whole $0.2$ . Average pooling gives dense gradients; max pooling gives sparse ones.

Exercise 3.3

Two consecutive $2 \times 2$ , stride-2 pooling layers are applied to an input. A single output neuron of the second pooling layer looks back at how many original input pixels? (This is its Receptive Field.)

Recall Solution

One 2nd-layer neuron summarizes a $2 \times 2$ block of 1st-layer neurons. Each of those summarizes a $2 \times 2$ block of the input. So along one axis it spans $2 \times 2 = 4$ pixels, and over 2D: $4 \times 4 = 16 input pixels.$ Each pooling stage doubles the receptive-field width, which is exactly why stacking pooling makes deep neurons "see" more of the image.

Level 4 — Synthesis

Exercise 4.1

Design a pooling stage. You have a $28 \times 28$ feature map and want the output to be exactly $14 \times 14$ using a $2 \times 2$ window. What stride do you choose, and prove it works.

Recall Solution

Try $s = 2$ : $⌊ \frac{28 - 2}{2} ⌋ + 1 = ⌊ 13 ⌋ + 1 = 14. ✓$ Stride $= 2$ . This is standard non-overlapping halving. (Stride $1$ would give $27$ ; stride $3$ would give $⌊ 26/3 ⌋ + 1 = 9$ — too small.)

Exercise 4.2

You are told a network replaces its final flatten-plus-dense block with Global Average Pooling. The last feature map is $512$ channels, each $7 \times 7$ . What is the output vector length, and how many learnable parameters did this pooling introduce?

Recall Solution

Global average pooling collapses each entire channel to its single mean value. So each of the $512$ channels becomes one number: $output length = 512.$ Learnable parameters introduced by the pooling: $0$ (pooling is parameter-free). This is precisely why GAP shrinks model size versus a dense layer, which would need $512 \times 7 \times 7 = 25088$ inputs times its output units in weights.

Exercise 4.3

Overlapping pooling: a $5 \times 5$ map, $3 \times 3$ window, stride $1$ . Find the output size and the overlap (in pixels) between two horizontally adjacent windows.

Recall Solution

Output size: $⌊ \frac{5 - 3}{1} ⌋ + 1 = 2 + 1 = 3 \Rightarrow 3 \times 3.$ Overlap between neighbours $= p - s = 3 - 1 = 2$ pixels. Adjacent windows start one pixel apart but each is $3$ wide, so they share $2$ columns. This is the "overlapping pooling" from the parent note's Mistake 2.

Level 5 — Mastery

Exercise 5.1

A feature map contains one strong activation of value $100$ surrounded by zeros in a $2 \times 2$ window: ${100, 0, 0, 0}$ . Compare how max vs average pooling represent this "sharp feature," and explain why max pooling is preferred for detecting sparse, localized features like edges.

Recall Solution

Max: output $= 100$ . The feature's presence and strength survive intact.
Average: $(100 + 0 + 0 + 0) /4 = 25$ . The single strong signal is diluted 4× by its zero neighbours.

For sparse detectors (an edge fires in only a few pixels), max pooling reports "yes, the feature is here, at full strength," giving Translation Invariance: it does not matter which of the four cells fired, the output is $100$ either way. Average pooling would report a weak $25$ and would change if the feature spread across more cells — worse for crisp detection. This is why architectures like VGG Network use max pooling between conv blocks.

Exercise 5.2

Prove that for average pooling, the routed-back gradient over one window always sums to exactly the incoming gradient $g = \partial L / \partial Y$ , for any $p$ .

Recall Solution

Each of the $p^{2}$ input cells in the window receives $\frac{1}{p ^{2}} g$ . Summing over all cells: $\sum_{k = 1}^{p^{2}} \frac{1}{p ^{2}} g = p^{2} \cdot \frac{1}{p ^{2}} g = g .$ So the total is conserved — average pooling neither amplifies nor loses gradient magnitude, it merely spreads it. (The same conservation held for max pooling in Exercise 3.1, but there all of $g$ went to a single cell.)

Exercise 5.3

A $9 \times 9$ map goes through $3 \times 3$ pooling, stride $3$ , then $2 \times 2$ pooling, stride $2$ . Find the final spatial size. Note any pixels dropped by a floor.

Recall Solution

Stage 1: $⌊(9 - 3) /3 ⌋ + 1 = ⌊ 2 ⌋ + 1 = 3 \Rightarrow 3 \times 3$ . Starts $0, 3, 6$ cover all $9$ rows exactly — nothing dropped. Stage 2: input is $3 \times 3$ : $⌊(3 - 2) /2 ⌋ + 1 = ⌊ 0 ⌋ + 1 = 1 \Rightarrow 1 \times 1$ . Window start $0$ covers rows $0, 1$ ; row 2 is dropped by the floor. Final size: $1 \times 1$ . This mirrors how the tail of a deep net (e.g. before the classifier in a ResNet Architecture) collapses spatial dimensions down to a single vector.

Recall Quick self-check (clozes)

Max pooling routes its gradient to only the argmax cell (the winner). Average pooling routes its gradient to ==all $p^{2}$ cells equally, each getting $g / p^{2}$ ==. Number of learnable parameters in any pooling layer ::: Zero. Output size formula ::: $⌊(H_{in} - p) / s ⌋ + 1$ .

Related: Global Average Pooling · Feature Maps · Translation Invariance · 3.4.03 Pooling layers (max, average) (Hinglish)