Feature maps and receptive fields: Exercises — AI-ML deep dive

This page is your self-test workshop. Every problem below builds on Feature maps and receptive fields. We climb the ladder: L1 Recognition → L2 Application → L3 Analysis → L4 Synthesis → L5 Mastery. Each solution is hidden inside a collapsible callout — try it yourself first, then reveal.

Before we start, the two formulas we will lean on the whole way (both come straight from the parent note, no new symbols):

Formula The two tools you need

Feature-map size (how big the output grid is). Let $O$ = output spatial size (the side length of the output feature map, in pixels): $O = ⌊ \frac{I + 2 p - k}{s} ⌋ + 1$ where $I$ = input side length, $p$ = padding, $k$ = kernel size, $s$ = stride, and $⌊ \cdot ⌋$ means "round down to the nearest whole number" (you cannot place a filter at half a pixel).

Receptive field (how much of the original image one output neuron feels). For layer $l$ (the $l$ -th operation in the stack, counting from the input as layer $0$ ), with kernel size $k_{l}$ and stride $s_{l}$ : $R_{l} = R_{l - 1} + (k_{l} - 1) \cdot \prod_{i = 1}^{l - 1} s_{i}, R_{0} = 1$ Here $R_{l}$ = receptive field at layer $l$ , and the product $\prod_{i = 1}^{l - 1} s_{i} = s_{1} \cdot s_{2} \dots s_{l - 1}$ is the cumulative stride — how many input pixels one step in layer $l$ actually jumps across. Throughout this page we abbreviate this running product as $J$ (for "Jump"), so $J = \prod_{i = 1}^{l - 1} s_{i}$ .

Keep these two boxes open in your mind. Everything is bookkeeping on top of them.

Level 1 — Recognition

Can you read a formula and plug numbers in?

Exercise 1.1

Input image is $28 \times 28$ . You apply one $5 \times 5$ filter, stride $1$ , padding $0$ . What is the output feature-map height?

Recall Solution

We use $O = ⌊(I + 2 p - k) / s ⌋ + 1$ because the question asks for output size $O$ , and this is the only formula that turns $(I, p, k, s)$ into an output dimension.

Plug in $I = 28, p = 0, k = 5, s = 1$ : $O = ⌊ \frac{28 + 0 - 5}{1} ⌋ + 1 = ⌊ 23 ⌋ + 1 = 24$ Answer: $24$ . With no padding, the filter cannot reach the edges, so the map shrinks from $28$ to $24$ .

Exercise 1.2

A convolutional layer has $16$ filters, each $3 \times 3$ . The input is $32 \times 32 \times 3$ . Stride $1$ , padding $1$ . Give the full output tensor shape $(C_{out}, H_{out}, W_{out})$ .

Recall Solution

Channels come from the number of filters: each filter produces one feature map ("one opinion"), so $C_{out} = 16$ . The input's $3$ channels vanish — a filter reads all input channels at once and collapses them into a single map.

Spatial size uses the size formula with $I = 32, p = 1, k = 3, s = 1$ : $O = ⌊ \frac{32 + 2 ( 1 ) - 3}{1} ⌋ + 1 = ⌊ 31 ⌋ + 1 = 32$ A $3 \times 3$ conv with padding $1$ preserves the spatial size.

Answer: $(16, 32, 32)$ .

Exercise 1.3

What is $R_{0}$ , and in plain words what does it mean?

Recall Solution

$R_{0} = 1$ . It is the base case of the recursion: before any convolution, an "input pixel sees only itself" — its receptive field is a single pixel wide.

Level 2 — Application

Can you run the recursion across a real stack?

Exercise 2.1

Two stacked $3 \times 3$ convolutions, both stride $1$ , no pooling. Compute the receptive field $R_{2}$ .

Recall Solution

Layer 1: $R_{1} = R_{0} + (k_{1} - 1) \cdot \prod_{i = 1}^{0} s_{i}$ . The empty product is $1$ (i.e. $J = 1$ ), so $R_{1} = 1 + (3 - 1) \cdot 1 = 3.$ Layer 2: cumulative stride from layer 1 is $J = s_{1} = 1$ , so $R_{2} = R_{1} + (k_{2} - 1) \cdot J = 3 + (3 - 1) \cdot 1 = 5.$ Answer: $R_{2} = 5$ . Two $3 \times 3$ convs "see" a $5 \times 5$ patch — the classic reason VGG stacks small kernels.

Exercise 2.2

Three stacked $3 \times 3$ stride- $1$ convs. Compute $R_{3}$ using the uniform-layer shortcut and confirm by recursion.

Recall Solution

Shortcut (for $s = 1$ ): $R_{l} = 1 + l (k - 1)$ . With $l = 3, k = 3$ : $R_{3} = 1 + 3 (3 - 1) = 1 + 6 = 7.$ Check by recursion: $R_{1} = 3, R_{2} = 5, R_{3} = 5 + (3 - 1) \cdot 1 = 7.$ ✓ Answer: $R_{3} = 7$ . For stride- $1$ stacks, receptive field grows linearly in depth.

Exercise 2.3

Conv ( $3 \times 3$ , stride $1$ ) → MaxPool ( $2 \times 2$ , stride $2$ ). Compute the receptive field after pooling and report the cumulative stride $J$ at each step.

Recall Solution

Pooling counts as a layer with kernel $k_{p} = 2$ and stride $s_{p} = 2$ . We track $R$ and $J$ together. Conv1 ( $k = 3, s = 1$ ): the cumulative stride entering it is $J = 1$ , so $R_{1} = 1 + (3 - 1) \cdot 1 = 3.$ Now update the jump: $J \leftarrow J \cdot s_{1} = 1 \cdot 1 = 1$ . Pool ( $k_{p} = 2, s_{p} = 2$ ): the cumulative stride entering it is still $J = 1$ , so $R_{2} = R_{1} + (k_{p} - 1) \cdot J = 3 + (2 - 1) \cdot 1 = 4.$ Now update the jump: $J \leftarrow J \cdot s_{p} = 1 \cdot 2 = 2$ . Answer: $R_{2} = 4$ , with cumulative stride $J = 2$ after the pool. The pool window covers $2$ conv outputs (adding $1$ pixel of reach) and doubles $J$ , which will amplify any later layer.

Look at the figure — the pool merges two adjacent conv-outputs, widening the input region from $3$ to $4$ pixels.

Figure — Feature maps and receptive fields

Level 3 — Analysis

Can you track the cumulative-stride multiplier $J$ and the interplay of layers?

Exercise 3.1

Full stack: Conv1 ( $3 \times 3$ , $s = 1$ ) → Pool ( $2 \times 2$ , $s = 2$ ) → Conv2 ( $3 \times 3$ , $s = 1$ ). Compute $R_{3}$ and the final cumulative stride $J$ .

Recall Solution

Track $R$ and $J = \prod s_{i}$ together. Conv1: $R_{1} = 1 + (3 - 1) \cdot 1 = 3$ . Update $J = 1$ . Pool: $R_{2} = 3 + (2 - 1) \cdot 1 = 4$ . Update $J = 1 \cdot 2 = 2$ . Conv2: the cumulative stride entering Conv2 is $J = 2$ , so each of its $3$ taps jumps $2$ input pixels: $R_{3} = R_{2} + (k - 1) \cdot J = 4 + (3 - 1) \cdot 2 = 4 + 4 = 8.$ Answer: $R_{3} = 8$ , cumulative stride $J = 2$ . The pool's stride amplified Conv2's expansion — that's the exponential engine.

Exercise 3.2

Same as 3.1 but insert a second pool ( $2 \times 2$ , $s = 2$ ) after Conv2. Compute $R_{4}$ .

Recall Solution

Continue from Ex 3.1: after Conv2, $R_{3} = 8$ , cumulative stride $J = 2$ . Pool2 ( $k_{p} = 2, s_{p} = 2$ ): the cumulative stride entering it is $J = 2$ : $R_{4} = R_{3} + (k_{p} - 1) \cdot J = 8 + (2 - 1) \cdot 2 = 8 + 2 = 10.$ Cumulative stride becomes $J = 2 \cdot 2 = 4$ . Answer: $R_{4} = 10$ , cumulative stride $J = 4$ .

Exercise 3.3

Two identical downsampling blocks stacked. Block = Conv ( $3 \times 3$ , $s = 1$ ) → Pool ( $2 \times 2$ , $s = 2$ ). Compute the receptive field after both blocks and note the growth pattern.

Recall Solution

Block 1:

Conv: $R = 1 + 2 \cdot 1 = 3$ , then $J = 1$ .
Pool: $R = 3 + 1 \cdot 1 = 4$ , then $J = 2$ .

Block 2:

Conv: $R = 4 + (3 - 1) \cdot 2 = 4 + 4 = 8$ , then $J = 2$ .
Pool: $R = 8 + (2 - 1) \cdot 2 = 8 + 2 = 10$ , then $J = 4$ .

Answer: $R = 10$ after two blocks. Compare with one block ( $R = 4$ ): the jump $4 \to 10$ is super-linear, because each pool doubles the multiplier $J$ that the next conv rides on.

Level 4 — Synthesis

Can you combine dilation, size, and receptive-field reasoning to design?

Exercise 4.1

A $3 \times 3$ dilated conv with dilation $d = 3$ . First find its effective kernel size $k_{eff}$ , then its single-layer receptive field.

Recall Solution

Dilation inserts $d - 1$ gaps between filter taps, so $k_{eff} = k + (k - 1) (d - 1) = 3 + (3 - 1) (3 - 1) = 3 + 4 = 7.$ As layer 1: $R_{1} = 1 + (k_{eff} - 1) \cdot 1 = 1 + 6 = 7.$ Answer: $k_{eff} = 7$ , $R_{1} = 7$ . A dilated $3 \times 3$ sees as wide as a plain $7 \times 7$ — but uses only $9$ weights, not $49$ .

Exercise 4.2

Design goal: reach a receptive field of at least $15$ using three stride- $1$ $3 \times 3$ layers with dilations $1, 2, 4$ . Verify it works.

Recall Solution

Compute $k_{eff}$ per layer, then run the recursion (cumulative stride stays $J = 1$ since every $s = 1$ ):

Layer 1 $d = 1$ : $k_{eff} = 3 + 2 \cdot 0 = 3$ . $R_{1} = 1 + 2 \cdot 1 = 3$ .
Layer 2 $d = 2$ : $k_{eff} = 3 + 2 \cdot 1 = 5$ . $R_{2} = 3 + (5 - 1) \cdot 1 = 7$ .
Layer 3 $d = 4$ : $k_{eff} = 3 + 2 \cdot 3 = 9$ . $R_{3} = 7 + (9 - 1) \cdot 1 = 15$ .

Answer: $R_{3} = 15 \geq 15$ . ✓ Exponentially-growing dilation ( $1, 2, 4$ ) reaches $15$ with zero downsampling, keeping full resolution — the core trick of dense-prediction networks.

Exercise 4.3

You need output spatial size equal to input ( $I = 100$ ) with a $7 \times 7$ stride- $1$ conv. What padding $p$ achieves this?

Recall Solution

Set $O = I$ in the size formula. First note we can drop the floor here: with stride $s = 1$ , the quantity $I + 2 p - k$ is a whole number, so $⌊(I + 2 p - k) /1 ⌋ = I + 2 p - k$ exactly — the floor only matters when $s > 1$ can create fractions. So: $100 = (100 + 2 p - 7) + 1 \Rightarrow 100 = 94 + 2 p \Rightarrow 2 p = 6 \Rightarrow p = 3.$ Answer: $p = 3$ . The "same-padding" rule for stride $1$ : $p = (k - 1) /2 = (7 - 1) /2 = 3$ . ✓

Level 5 — Mastery

Can you reason about a whole architecture and its limits?

Exercise 5.1

A network: Conv1 ( $5 \times 5$ , $s = 1$ , $p = 2$ ) → Pool ( $2 \times 2$ , $s = 2$ ) → Conv2 ( $3 \times 3$ , $s = 1$ , $p = 1$ ) → Pool ( $2 \times 2$ , $s = 2$ ). Input $64 \times 64$ . Give (a) the final spatial size and (b) the receptive field of a final-layer neuron.

Recall Solution

(a) Spatial size, applying the size formula stage by stage:

Conv1: $O = ⌊(64 + 4 - 5) /1 ⌋ + 1 = 63 + 1 = 64$ .
Pool1: $O = ⌊(64 - 2) /2 ⌋ + 1 = 31 + 1 = 32$ .
Conv2: $O = ⌊(32 + 2 - 3) /1 ⌋ + 1 = 31 + 1 = 32$ .
Pool2: $O = ⌊(32 - 2) /2 ⌋ + 1 = 15 + 1 = 16$ . Final spatial size: $16 \times 16$ .

(b) Receptive field, tracking the cumulative-stride multiplier $J = \prod_{i} s_{i}$ introduced in the formula box:

Conv1: $R = 1 + (5 - 1) \cdot 1 = 5$ , then $J = 1$ .
Pool1: $R = 5 + (2 - 1) \cdot 1 = 6$ , then $J = 1 \cdot 2 = 2$ .
Conv2: $R = 6 + (3 - 1) \cdot 2 = 6 + 4 = 10$ , then $J = 2$ .
Pool2: $R = 10 + (2 - 1) \cdot 2 = 12$ , then $J = 2 \cdot 2 = 4$ . Receptive field: $12 \times 12$ .

Answer: (a) $16 \times 16$ , (b) $R = 12$ . Each final neuron summarizes a $12 \times 12$ image patch into one value on a $16 \times 16$ grid.

Exercise 5.2

Push it: how many extra stride- $1$ $3 \times 3$ layers (each adds $(3 - 1) \cdot J$ with $J = 4$ ) would you append to the network of Ex 5.1 to make the receptive field cover the entire $64 \times 64$ input (i.e. $R \geq 64$ )?

Recall Solution

After Ex 5.1, $R = 12$ and cumulative stride $J = 4$ . Each appended stride- $1$ $3 \times 3$ layer adds: $Δ R = (3 - 1) \cdot J = 2 \cdot 4 = 8.$ We need $12 + 8 n \geq 64 \Rightarrow 8 n \geq 52 \Rightarrow n \geq 6.5$ . Since $n$ is a whole number, $n = 7$ . Check: $12 + 8 \cdot 7 = 12 + 56 = 68 \geq 64$ . ✓ (With $n = 6$ : $12 + 48 = 60 < 64$ , not enough.) Answer: $7$ extra layers.

Exercise 5.3 (Degenerate / limiting case)

A $1 \times 1$ convolution, any stride, any depth. What is its receptive field, and why is that the extreme case worth knowing?

Recall Solution

For $k = 1$ : the expansion term is $(k - 1) \cdot J = 0 \cdot J = 0$ at every layer. So $R_{l} = R_{l - 1} = \dots = R_{0} = 1$ forever. Answer: $R = 1$ at any depth. A $1 \times 1$ conv never widens the spatial view — it only mixes channels at a fixed pixel. It is the degenerate "zero-width-growth" case, and precisely why $1 \times 1$ convs are used purely for channel reshaping in deeper architectures.

Related: 3.4.01-Convolutional-layers · 3.4.02-Pooling-layers · 3.4.03-Padding-and-stride · 3.3.02-Activation-functions