Convolutional neural networks — convolution operation, pooling: Exercises

Intuition How to use this page

Work each problem before you open its solution. The problems climb a ladder: L1 Recognition (can you spot the right formula?) → L2 Application (can you turn the crank?) → L3 Analysis (can you reason about why?) → L4 Synthesis (can you combine pieces?) → L5 Mastery (can you design and defend a choice?). Every symbol here was built in the parent note; if a symbol feels new, re-read section 1 there first.

Quick symbol key (all from the parent):

$W_{in}$ = input width in pixels (a whole number, e.g. 32).
$k$ = kernel side length (a $3 \times 3$ kernel has $k = 3$ ).
$p$ = padding (extra rings of zeros added around the image, one number = one ring width).
$s$ = stride (how many pixels the kernel jumps each slide).
$C$ = number of input channels; $F$ = number of filters (= output channels).
$⌊ x ⌋$ = "floor": round down to the nearest whole number. $⌊ 3.9 ⌋ = 3$ .

The two workhorse formulas we will lean on all page:

Level 1 — Recognition

Problem 1.1

Which of the following operations has zero learnable parameters? (a) a $3 \times 3$ convolution, (b) $2 \times 2$ max-pooling, (c) a fully-connected layer, (d) the bias term.

Recall Solution 1.1

Answer: (b) max-pooling. What we did: recalled that pooling is a fixed function — it computes a max (or average) over a window and there is nothing to train inside it. A convolution has $k^{2} C + 1$ trainable numbers per filter; a fully-connected layer has one weight per input–output pair; the bias term is a parameter. Only pooling learns nothing.

Problem 1.2

State, from memory, the output-size formula and name each of its four inputs.

Recall Solution 1.2

$W_{o u t} = ⌊ \frac{W _{in} - k + 2 p}{s} ⌋ + 1$ $W_{in}$ = input width, $k$ = kernel size, $p$ = padding, $s$ = stride. Mnemonic (parent note): "I-Kicked-2-Pads-per-Stride-plus-one."

Problem 1.3

Input width $W_{in} = 7$ , kernel $k = 3$ , padding $p = 0$ , stride $s = 1$ . Compute $W_{o u t}$ .

Recall Solution 1.3

$W_{o u t} = ⌊ \frac{7 - 3 + 0}{1} ⌋ + 1 = ⌊ 4 ⌋ + 1 = 5$ What it looks like: a $3$ -wide kernel can start at columns $0, 1, 2, 3, 4$ — that's 5 positions, so 5 outputs. The $- k$ chops off the room the kernel body needs; the $+ 1$ puts back the very first position (column 0), which the subtraction never counted.

Level 2 — Application

Problem 2.1

Input $W_{in} = 32$ , kernel $k = 5$ , padding $p = 2$ , stride $s = 1$ . Find $W_{o u t}$ and say in words what this padding choice achieves.

Recall Solution 2.1

$W_{o u t} = ⌊ \frac{32 - 5 + 2 ( 2 )}{1} ⌋ + 1 = ⌊ \frac{31}{1} ⌋ + 1 = 32$ Why this padding: with $p = 2$ we added two zero-rings, exactly enough to cancel the $- k$ shrink for a $5 \times 5$ kernel at stride 1 ( $2 p = k - 1$ ). This is "same" padding — output size equals input size. Useful when you want to stack many conv layers without the map vanishing.

Problem 2.2

Compute the full $2 \times 2$ output of convolving $I = 210132014, K = [1 - 1 - 1 1], b = 0$ with stride 1, no padding.

Recall Solution 2.2

Output size: $⌊(3 - 2) /1 ⌋ + 1 = 2$ , so a $2 \times 2$ map. Overlay $K$ on each $2 \times 2$ window and sum element-wise products (cross-correlation, parent §1).

$S (0, 0) = 2 (1) + 1 (- 1) + 1 (- 1) + 3 (1) = 2 - 1 - 1 + 3 = 3$
$S (0, 1) = 1 (1) + 0 (- 1) + 3 (- 1) + 1 (1) = 1 - 0 - 3 + 1 = - 1$
$S (1, 0) = 1 (1) + 3 (- 1) + 0 (- 1) + 2 (1) = 1 - 3 - 0 + 2 = 0$
$S (1, 1) = 3 (1) + 1 (- 1) + 2 (- 1) + 4 (1) = 3 - 1 - 2 + 4 = 4$ $S = [30 - 1 4]$ What it looks like: this kernel fires positively when the two diagonal pixels are large and the two anti-diagonal pixels are small — a checkerboard / corner detector.

Problem 2.3

Max-pool the map $S$ below with a $2 \times 2$ window, stride 2. Then average-pool it. $S = 1460321720311521$

Recall Solution 2.3

Stride 2 with a $2 \times 2$ window ⇒ non-overlapping $2 \times 2$ blocks. Output $⌊(4 - 2) /2 ⌋ + 1 = 2$ , a $2 \times 2$ map. Max-pool (keep the loudest in each block):

TL ${1, 3, 4, 2} \to 4$ , TR ${2, 1, 0, 5} \to 5$
BL ${6, 1, 0, 7} \to 7$ , BR ${3, 2, 1, 1} \to 3$ $P_{ma x} = [4753]$ Average-pool (mean of each block):
TL $(1 + 3 + 4 + 2) /4 = 2.5$ , TR $(2 + 1 + 0 + 5) /4 = 2$
BL $(6 + 1 + 0 + 7) /4 = 3.5$ , BR $(3 + 2 + 1 + 1) /4 = 1.75$ $P_{a v g} = [2.5 3.5 2 1.75]$

Level 3 — Analysis

Problem 3.1

A conv layer takes a $128 \times 128$ image with $C = 3$ channels, uses $F = 64$ filters of size $3 \times 3$ . How many learnable parameters? Now the image becomes $512 \times 512$ — how many parameters now? Explain the result.

Recall Solution 3.1

$# params = F (k^{2} C + 1) = 64 (3^{2} \cdot 3 + 1) = 64 (27 + 1) = 64 \cdot 28 = 1792$ For the $512 \times 512$ image: still 1792. The formula $F (k^{2} C + 1)$ contains no $W_{in}$ . Why: the same kernel is slid over the whole image (weight sharing), so making the image bigger just means more slides, not more weights. This size-independence is the defining advantage of convolution over a fully-connected layer.

Problem 3.2

Take the map $S = [0018]$ (values are activations). Shift the "8" one cell left to get $S^{'} = [0810]$ . Apply a single $2 \times 2$ max-pool to each. What do you observe, and what property does this demonstrate?

Recall Solution 3.2

$max (S) = 8$ and $max (S^{'}) = 8$ . Both pool to the same value 8. What this shows: local translation invariance — the feature (the strong activation "8") moved by one pixel, yet the pooled output did not change. Max-pooling reports "the feature is present in this window" without caring exactly where in the window. See the figure.

Problem 3.3

You stack two stride-1 $3 \times 3$ conv layers (no padding). A single neuron in the second layer's output — how many input pixels does it ultimately depend on (its receptive field width)?

Recall Solution 3.3

One $3 \times 3$ conv gives a neuron a receptive field of width 3. Feeding that into a second $3 \times 3$ conv: the second neuron combines 3 first-layer neurons, and those 3 neighbours cover input columns $[0, 2], [1, 3], [2, 4]$ — together columns $0$ through $4$ , i.e. width 5. General rule for stacked stride-1 conv: receptive field grows by $(k - 1)$ per layer, so after two $3 \times 3$ layers it is $3 + (3 - 1) = 5$ . Two small kernels see as far as one $5 \times 5$ kernel, with fewer parameters — that's why deep stacks of small kernels are preferred.

Level 4 — Synthesis

Problem 4.1 — trace a small pipeline

An aerospace crack-inspection net receives a $64 \times 64$ grayscale patch ( $C = 1$ ). It applies:

Conv: $F = 16$ , $k = 3$ , $p = 1$ , $s = 1$
Max-pool: $2 \times 2$ , $s = 2$
Conv: $F = 32$ , $k = 3$ , $p = 1$ , $s = 1$
Max-pool: $2 \times 2$ , $s = 2$

Give the spatial size and channel count after each stage, and the total conv parameters.

Recall Solution 4.1

Track $(W \times H \times channels)$ . Start: $64 \times 64 \times 1$ .

Stage 1 conv ( $k = 3, p = 1, s = 1$ = "same"): $⌊(64 - 3 + 2) /1 ⌋ + 1 = 64$ . → $64 \times 64 \times 16$ .
Stage 2 pool ( $2 \times 2, s = 2$ ): $⌊(64 - 2) /2 ⌋ + 1 = 32$ . → $32 \times 32 \times 16$ .
Stage 3 conv (same padding): $32 \times 32 \times 32$ .
Stage 4 pool: $⌊(32 - 2) /2 ⌋ + 1 = 16$ . → $16 \times 16 \times 32$ .

Parameters (pooling has none):

Conv 1: $16 (3^{2} \cdot 1 + 1) = 16 \cdot 10 = 160$
Conv 3: $32 (3^{2} \cdot 16 + 1) = 32 \cdot 145 = 4640$
Total $= 160 + 4640 = 4800$ .

Why "same"-padding matters here: it lets the conv layers keep the spatial size fixed, so all the shrinking is done by pooling in a controlled, predictable $\div 2$ pattern — easy to reason about.

Problem 4.2 — build the equivalent FC count for contrast

For Stage 1 above, how many weights would a fully-connected layer need to map every input pixel to every output unit (ignore bias)? Compare to the conv's 160.

Recall Solution 4.2

Input has $64 \times 64 \times 1 = 4096$ units. Output has $64 \times 64 \times 16 = 65536$ units. Fully-connected weights $= 4096 \times 65536 = 268, 435, 456 \approx 2.68 \times 1 0^{8}$ . Conv used 160. Ratio $\approx 1.68 \times 1 0^{6}$ — the conv is over a million times leaner. Why: locality (each output only touches a $3 \times 3$ neighbourhood) + weight sharing (one kernel reused everywhere) together collapse that quarter-billion matrix into 160 numbers. This is the parent's "convolution is a constrained fully-connected layer" made concrete.

Level 5 — Mastery

Problem 5.1 — hit a target output size

You must design one conv layer that takes a $224 \times 224$ input and produces a $112 \times 112$ output using a $k = 3$ kernel. Choose padding $p$ and stride $s$ , and prove it works.

Recall Solution 5.1

Try stride $s = 2$ (to roughly halve). Solve for $p$ that gives exactly 112: $112 = ⌊ \frac{224 - 3 + 2 p}{2} ⌋ + 1 \Rightarrow ⌊ \frac{221 + 2 p}{2} ⌋ = 111$ We need $\frac{221 + 2 p}{2} \in [111, 112)$ ⇒ $221 + 2 p \in [222, 224)$ ⇒ $2 p \in [1, 3)$ ⇒ $p = 1$ . Check with $p = 1, s = 2$ : $⌊(224 - 3 + 2) /2 ⌋ + 1 = ⌊ 223/2 ⌋ + 1 = ⌊ 111.5 ⌋ + 1 = 111 + 1 = 112.$ ✓ Answer: $p = 1, s = 2$ . Why stride 2, not padding tricks: halving is fundamentally a stride job (stride sets the rate of down-sampling); padding then fine-tunes the edge so the floor lands exactly on 112.

Problem 5.2 — defend max vs average for a crack detector

For classifying whether a composite panel image contains a crack (yes/no), argue which pooling — max or average — better preserves the decisive signal, and give the failure mode of the wrong one.

Recall Solution 5.2

Choose max-pooling. A crack is a sparse, high-contrast feature: a thin bright line covering a tiny fraction of the panel. A conv filter tuned to that line produces one or a few large activations surrounded by near-zero background.

Max-pool keeps that large activation intact — it answers "did the crack feature appear at all in this window?", which is exactly the yes/no question.
Average-pool divides the strong activation by the whole window's cell count, drowning a single hot pixel in a sea of zeros. A real crack could be averaged down below the decision threshold — a missed defect, the worst error in inspection. When average wins instead: tasks about overall magnitude / texture (e.g. estimating mean surface roughness), where the smoothing is a feature, not a bug.

Problem 5.3 — the flip question, defended

A colleague hand-codes the Sobel edge kernel from a signal-processing textbook and drops it into a Conv2d layer, expecting mathematically exact convolution. Where might results differ, and does it matter for a learned network?

Recall Solution 5.3

Conv2d computes cross-correlation (no kernel flip), while the textbook's true convolution flips the kernel by 180° first (parent §1). For a hand-designed asymmetric kernel like Sobel, cross-correlation applies the mirrored filter — the detected edge orientation can invert. Fix for hand-coded kernels: pre-flip the kernel (rotate 180°) before loading it. For a learned network: it does not matter at all — gradient descent simply learns whichever orientation of weights minimizes loss, i.e. it learns the flipped values automatically. The flip is only a concern when you inject a fixed, meaningful kernel.

Recall One-line self-check before you leave

Cover the answers: $W_{o u t} = ?$ ::: $⌊(W_{in} - k + 2 p) / s ⌋ + 1$ Params of a conv layer ::: $F (k^{2} C + 1)$ , image-size independent Pooling params ::: zero Best pool for sparse crack signal ::: max

Connections

Parent topic
Padding and stride — the $p$ and $s$ every L2–L5 problem tuned.
Feature maps and receptive fields — the receptive-field growth in Problem 3.3.
Translation equivariance vs invariance — the pooling-invariance demo in Problem 3.2.
Image classification for aerospace inspection — the crack-detector design in L4/L5.
Fully-connected neural networks — the $2.68 \times 1 0^{8}$ -weight contrast in Problem 4.2.
Backpropagation — how the 4800 learnable params of Problem 4.1 actually get trained.