Convolution operation and filters: Exercises — AI-ML deep dive

These graded exercises take you from recognizing the convolution formula all the way to building your own layer configurations. Every problem has a full solution hidden inside a collapsible callout — try first, then reveal. This page is the practice companion to Convolution operation and filters.

Prerequisite links you may want open: 2.2.05-Receptive-fields, 3.4.02-Pooling-layers, 3.3.01-Backpropagation, 3.4.03-CNN-architectures.

Level 1 — Recognition

Exercise 1.1 — Read the formula

State, in plain words, what each symbol means in

(I * K) [i, j] = m = 0 \sum k - 1 n = 0 \sum k - 1 I [i + m, j + n] \cdot K [m, n] .

Recall Solution

$I$ is the input image (a grid of numbers).
$K$ is the filter/kernel, a small $k \times k$ grid of weights.
$(i, j)$ is the top-left corner of the window we currently sit on.
$m, n$ index inside the window: they run $0, 1, \dots, k - 1$ , so they visit every cell of the filter.
$I [i + m, j + n]$ is the image pixel under filter cell $(m, n)$ .
The double sum multiplies each filter weight by the pixel beneath it and adds up all $k^{2}$ products.
The result is one number: the template-matching score at that location.

Exercise 1.2 — Output size, plug and chug

An input is $28 \times 28$ , filter $k = 3$ , stride $s = 1$ , padding $p = 0$ . What is the output height $H_{o u t}$ ?

Recall Solution

$H_{o u t} = ⌊ \frac{H + 2 p - k}{s} ⌋ + 1 = ⌊ \frac{28 + 0 - 3}{1} ⌋ + 1 = 25 + 1 = 26.$ Output is $26 \times 26$ . The image shrank by $k - 1 = 2$ because we used no padding.

Exercise 1.3 — Name the filter

Which pattern does this filter detect?

- 1 01 - 1 01 - 1 01

Recall Solution

This is a horizontal edge detector. The top row has negative weights, the bottom row positive. It fires strongly (large positive score) when the region below is bright and the region above is dark — i.e. a horizontal boundary from dark-on-top to bright-on-bottom.

Level 2 — Application

Exercise 2.1 — Compute one output cell

Input patch (top-left $3 \times 3$ of some image) and filter:

I_{patch} = 410052236, K = 111000 - 1 - 1 - 1 .

Compute the convolution score. Look at Figure s01 for the overlay.

Figure — Convolution operation and filters

Recall Solution

Multiply each cell by the filter cell above it (the middle column is $\times 0$ , so ignore it): $4 (1) + 0 (0) + 2 (- 1) + 1 (1) + 5 (0) + 3 (- 1) + 0 (1) + 2 (0) + 6 (- 1) .$ $= 4 - 2 + 1 - 3 + 0 - 6 = - 6.$ Score $= - 6$ . This filter is a left-minus-right vertical edge detector; the negative value means the right side of this patch is brighter than the left.

Exercise 2.2 — Output size with stride and padding

Input $32 \times 32$ , $k = 5$ , stride $s = 2$ , padding $p = 2$ . Find $H_{o u t}$ .

Recall Solution

$H_{o u t} = ⌊ \frac{32 + 2 ( 2 ) - 5}{2} ⌋ + 1 = ⌊ \frac{31}{2} ⌋ + 1 = 15 + 1 = 16.$ Output is $16 \times 16$ . Stride 2 roughly halved the size; padding 2 kept the arithmetic clean.

Exercise 2.3 — Full feature map

Input $I$ and vertical-edge filter $K$ :

I = 1302211002111021, K = - 1 - 1 - 1 000111, s = 1, p = 0.

Compute the full $2 \times 2$ output. Figure s02 shows the two slide positions.

Recall Solution

Output size: $⌊(4 - 3) /1 ⌋ + 1 = 2$ , so $2 \times 2$ . Middle column of $K$ is $0$ , so we only use left column ( $- 1$ ) and right column ( $+ 1$ ). $(0, 0)$ , window rows 0–2, cols 0–2: $- (1 + 3 + 0) + (0 + 2 + 1) = - 4 + 3 = - 1.$ $(0, 1)$ , cols 1–3: $- (2 + 1 + 1) + (1 + 0 + 2) = - 4 + 3 = - 1.$ $(1, 0)$ , rows 1–3, cols 0–2: $- (3 + 0 + 2) + (2 + 1 + 1) = - 5 + 4 = - 1.$ $(1, 1)$ , rows 1–3, cols 1–3: $- (1 + 1 + 0) + (0 + 2 + 1) = - 2 + 3 = 1.$ $Output = [- 1 - 1 - 1 1] .$

Level 3 — Analysis

Exercise 3.1 — Padding to preserve size

You have $k = 7$ , stride $s = 1$ . What padding $p$ keeps the output the same size as the input?

Recall Solution

Set $H_{o u t} = H$ in $H = ⌊(H + 2 p - k) /1 ⌋ + 1$ : $H - 1 = H + 2 p - k ⟹ 2 p = k - 1 ⟹ p = \frac{k - 1}{2} = \frac{7 - 1}{2} = 3.$ $p = 3$ . General rule for stride-1 "same" padding: $p = (k - 1) /2$ , which needs $k$ odd.

Exercise 3.2 — Parameter count

A conv layer takes an input of depth $C_{in} = 3$ (RGB), uses $N = 64$ filters of size $5 \times 5$ , and has one bias per filter. How many learnable parameters?

Recall Solution

Each filter is $5 \times 5 \times C_{in} = 5 \times 5 \times 3 = 75$ weights, plus 1 bias $= 76$ per filter. Total $= 76 \times 64 = 4864$ parameters. Compare to a fully connected layer on a $32 \times 32 \times 3$ image feeding 64 outputs: $32 \cdot 32 \cdot 3 \cdot 64 = 196, 608$ . Convolution's weight sharing is why it wins.

Exercise 3.3 — Receptive field growth

Two stacked $3 \times 3$ conv layers (stride 1, no dilation). What is the receptive field of one output neuron in the second layer, measured on the original input? Connect this to 2.2.05-Receptive-fields.

Recall Solution

One $3 \times 3$ layer sees $3 \times 3$ . Stacking a second $3 \times 3$ on top: each of those $3 \times 3$ positions itself sees $3 \times 3$ , and they overlap. The formula for stacked stride-1 layers: $R F = 1 + \sum_{i} (k_{i} - 1) = 1 + (3 - 1) + (3 - 1) = 5.$ Receptive field $= 5 \times 5$ . Two cheap $3 \times 3$ layers ( $2 \times 9 = 18$ weights) cover the same window as one $5 \times 5$ layer ( $25$ weights) — with more nonlinearity and fewer parameters.

Level 4 — Synthesis

Exercise 4.1 — Design a shape-matching stack

You must map a $224 \times 224 \times 3$ input to a $112 \times 112 \times 32$ feature map in one conv layer. Give a valid $(k, s, p, N)$ .

Recall Solution

Depth $32$ means $N = 32$ filters. For the spatial halving $224 \to 112$ , try $k = 3$ , $s = 2$ : $H_{o u t} = ⌊ \frac{224 + 2 p - 3}{2} ⌋ + 1 = 112 ⟹ ⌊ \frac{221 + 2 p}{2} ⌋ = 111.$ Take $p = 1$ : $⌊ 223/2 ⌋ = 111$ . ✓ So $(k, s, p, N) = (3, 2, 1, 32)$ works. (Also valid: $k = 4, s = 2, p = 1$ .) Depth of each filter is $3$ to match the RGB input, giving $3 \cdot 3 \cdot 3 \cdot 32 = 864$ weights plus 32 biases.

Exercise 4.2 — Multi-channel single output

Input $2 \times 2 \times 2$ (two channels), filter $2 \times 2 \times 2$ :

I_{c = 0} = [1021], I_{c = 1} = [3102]; K_{c = 0} = [1001], K_{c = 1} = [0110] .

Compute the single output value (valid convolution gives $1 \times 1$ ). See Figure s03.

Recall Solution

Sum over channels, then over the window: $ch 0 : 1 (1) + 2 (0) + 0 (0) + 1 (1) = 2.$ $ch 1 : 3 (0) + 0 (1) + 1 (1) + 2 (0) = 1.$ Total $= 2 + 1 = 3$ . The two channel results are added, not stacked — a multi-channel filter collapses all input depth into one number per position.

Level 5 — Mastery

Exercise 5.1 — Dilation and effective kernel

A $3 \times 3$ filter with dilation $d = 2$ has gaps inserted between its cells. What is its effective kernel size, and what output size does it give on a $10 \times 10$ input with $s = 1$ , $p = 0$ ?

Recall Solution

Effective size: $k_{eff} = d (k - 1) + 1 = 2 (3 - 1) + 1 = 5$ . So it covers a $5 \times 5$ region using only $9$ weights. Output: $⌊(10 - 5) /1 ⌋ + 1 = 6$ , i.e. $6 \times 6$ . Dilation grows the receptive field (2.2.05-Receptive-fields) without adding parameters — the trick behind atrous/dilated convolutions.

Exercise 5.2 — Cost of separable convolution

Compare multiply-adds for one output position (across all channels) between a standard $3 \times 3$ conv with $C_{in} = 16$ , $N = 32$ , and a depthwise-separable version (3.4.10-Depthwise-separable-convolutions).

Recall Solution

Standard: each output channel needs $3 \cdot 3 \cdot 16$ mult-adds; times $32$ outputs: $3 \cdot 3 \cdot 16 \cdot 32 = 4608.$ Separable = depthwise ( $3 \times 3$ per input channel) + pointwise ( $1 \times 1$ mixing): $depthwise 3 \cdot 3 \cdot 16 + pointwise 16 \cdot 32 = 144 + 512 = 656.$ Ratio $= 656/4608 \approx 0.142$ : about a 7× reduction in compute for this configuration. That's why mobile CNNs use them.

Exercise 5.3 — Backprop sanity check

In the forward pass an output element is $y = \sum_{m, n} I [i + m, j + n] K [m, n]$ . During 3.3.01-Backpropagation, what is $\partial y / \partial K [m, n]$ , and what does it tell you about learning?

Recall Solution

$\frac{\partial y}{\partial K [ m , n ]} = I [i + m, j + n] .$ The gradient on a filter weight is just the input pixel it multiplied. Summed over every position where the filter fired (because of weight sharing), the update becomes a convolution of the input with the upstream gradient. So the same weight collects evidence from the whole image — this is exactly why a filter learns one consistent feature everywhere. Bright inputs where the filter helped → large positive gradient → weight strengthens.