3.4.3 · D5Convolutional Neural Networks
Question bank — Pooling layers (max, average)
This bank hunts the misconceptions around pooling — the ideas that feel true but break under a careful look. Read each prompt, cover the answer, commit to a verdict out loud, then reveal. Every answer gives you the reasoning, not just the verdict.
The four symbols you need (read this first)
Every trap below leans on the same handful of letters. Let us define each one and pin it to the picture before we use it.
Look at the figure. The blue box is the pool window (). It sits over the top-left corner, produces one number, then slides right by the stride (yellow arrow) to a new, non-overlapping spot. That sliding is literally all pooling does.
Why the output-size formula looks the way it does
The formula is not magic — you can build it by counting box positions.
The figure above shows max pooling (blue, keeps the peak) versus average pooling (green, keeps the mean) on the same window — the only difference between the two operations.
The last figure shows gradient routing: in max pooling the whole (red) flows back only to the winning cell; in average pooling it splits evenly into across all four cells (green). Keep these pictures in mind for the questions.
True or false — justify
A 2×2 max pooling with stride 2 halves both height and width for any even input.
True — with window , stride , the formula when is even, so a 16×16 map becomes 8×8.
Pooling layers contain trainable weights that the network learns by gradient descent.
False — max and average are fixed arithmetic operations with zero learnable parameters; the learning lives in the conv layers that feed them.
Average pooling and max pooling always produce the same output dimensions given the same window and stride.
True — output size depends only on via ; what you compute inside the window (max vs mean) never changes how many windows fit.
Max pooling makes a CNN completely invariant to any translation of the input.
False — it gives local, approximate translation invariance: shifts within a pooling window are absorbed, but a shift larger than the window still moves the output.
Applying pooling increases the receptive field of later neurons.
True — because each output neuron now summarises a patch, downstream neurons "see" a larger slice of the original input than they would without downsampling.
Stride must always equal the pool size so windows never overlap.
False — you may set for overlapping pooling (e.g. 3×3 window, stride 2); windows then overlap by pixels, which can help accuracy at higher cost.
Average pooling passes gradient to only one input position per window.
False — that's max pooling. Average pooling spreads the incoming gradient equally to all positions, each receiving .
Global Average Pooling replacing the final dense layers reduces the parameter count.
True — GAP collapses each channel to a single mean with no weights, whereas a fully-connected layer over a flattened map has huge weight matrices; GAP also fights overfitting.
Pooling can never increase a value in the feature map.
True for both — max returns an existing element (so the window max, which is in the window), and average returns a mean, which lies between the min and max of the window. Neither invents a larger number.
With 'valid' padding, a pool with stride 2 on a input gives a output.
True —
'valid' means "no padding, drop partial windows," so and the last row/column is discarded.Spot the error
"For a 7×7 input with a 3×3 pool and stride 2, the output is 4×4 because 7/2 rounds up to 4."
Wrong reasoning and wrong answer — use , giving a 3×3 output; you subtract the window before dividing, not just divide the input.
"During backprop through max pooling, we split the output gradient equally among the four inputs of each 2×2 window."
That's the average-pooling rule. In max pooling the whole gradient goes to the single position that held the max in the forward pass; the other three get exactly 0.
"Pooling reduces overfitting because it adds a regularising penalty to the loss."
No penalty is added. It reduces overfitting by discarding spatial detail (fewer degrees of freedom to memorise exact positions), not by modifying the loss function.
"Average pooling is always inferior to max pooling since VGG and ResNet use max."
A popularity fallacy — average pooling often wins for global pooling, texture features, and segmentation; modern nets like MobileNet/EfficientNet lean on average pooling heavily.
"Because pooling has no parameters, gradients don't need to pass through it during training."
Wrong — no parameters means nothing to update inside pooling, but gradients absolutely flow through it to reach the earlier conv layers, following the max/average routing rules.
"Max pooling of a 4×4 map with a 2×2 window, stride 1, gives a 2×2 output."
With stride 1 the windows overlap: , so the output is 3×3, not 2×2 — smaller stride means more windows fit.
"With 'same' padding the output of pooling is always the same size as the input."
Only when stride —
'same' pads so no data is dropped, but with stride the output still shrinks by roughly a factor of (e.g. ).Why questions
Why does max pooling give sparse gradients while average pooling gives dense gradients?
In the forward pass only the max element influenced the output, so by the chain rule only its path carries gradient; averaging used all elements equally, so all four paths carry an equal share.
Why is max pooling favoured for object detection/classification but average pooling for texture?
Max answers "is this feature present anywhere in the patch?" — good for spotting a sharp cue; average answers "how much of this feature is here on the whole?" — good for smooth, distributed patterns like texture.
Why does pooling make the network cheaper in deeper layers?
Halving height and width quarters the number of spatial positions, so every following conv layer processes far fewer locations, cutting computation and memory downstream.
Why can a small object shift leave a max-pooled output unchanged?
If the feature's strongest activation moves but stays inside the same window, the max picks the same peak value — the shift is absorbed, giving local translation invariance.
Why might overlapping pooling () improve accuracy?
Overlapping windows retain more spatial context and give a denser, less abrupt downsampling, so less information is lost between adjacent output cells — at the cost of extra computation.
Why does the output-size formula subtract before dividing by rather than just dividing by ?
Because the window occupies columns, its left edge can only reach ; you count legal left-edge positions over that shortened span, then add 1 for the first position.
Edge cases
What does average pooling output for a window of all zeros?
Exactly — the mean of zeros is zero; average pooling passes a "no feature" region through as a clean zero.
What does max pooling output when every value in a window is negative?
The least negative value (the maximum, e.g. over ) — max still just picks the largest, even if all are below zero.
If the pool window is 1×1 with stride 1, what happens?
Nothing changes — each window is a single element, so both max and average return the input unchanged (an identity map).
If , , with 'valid' padding, what happens to the leftover row/column?
, so output is 2×2 and the last row/column is dropped — a partial window that doesn't fully fit is simply ignored.
What if you do pad ('same' mode) instead of dropping that leftover?
The framework adds a border of zeros (or edge-replicated values) so the partial window is completed, and that final window pools over real data plus the padding — no row is lost, but the padded cells contribute their fill value to the max/mean.
What happens when the pool window is larger than the input, , with 'valid' padding?
— no full window fits, so
'valid' pooling produces an empty (undefined) output; you must pad or use global pooling instead.What happens to that same case under global pooling?
Global pooling ignores and entirely — it pools over the whole map at once, collapsing any size to a single number per channel, so it always works.
For a window where two positions tie for the maximum, where does the gradient go in max pooling?
By convention it goes to one position (typically the first found by the implementation); the tie is broken deterministically so a single path carries the gradient, not both.
What is the output of GAP applied to a single channel?
A single scalar — the mean of all values — collapsing the whole spatial map for that channel into one number.
Does changing max pooling to average pooling change the number of output channels?
No — pooling operates independently per channel and never mixes them, so the channel count is identical; only the per-channel spatial summary differs.
Recall Three anchors to remember
Pooling learns nothing ::: it has zero trainable parameters; intelligence sits in the conv layers before it. Stride and window size are independent ::: overlap is allowed when , and output size is . Gradient routing differs ::: max sends the gradient to the single winning position; average splits it evenly across all .