Question bank — CNN architecture design

True. The first 3×3 sees a 3×3 window; the second 3×3 sees a 3×3 window of those, so it reaches $(3-1)+(3-1)+1=5$ pixels back — a 5×5 view, exactly like one 5×5 conv.

False. For $C$ channels, two 3×3s cost $C^2(9+9)=18C^2$ while one 5×5 costs $C^2\cdot 25$ — the stack is ~28% cheaper, and it adds a second nonlinearity.

False. An identity skip ($\mathbf{H}(\mathbf{x})=\mathbf{F}(\mathbf{x})+\mathbf{x}$) adds a plain addition, which has zero weights. Only a projection skip (used when shapes change) adds a small 1×1 conv.

False. This is the exact observation that motivated ResNet: past ~20 plain layers, training error rose, because gradients degrade — deeper is not automatically more trainable.

True in the trivial sense — pooling has no parameters at all. It has a fixed rule (take the max), so it adds capacity of zero while shrinking spatial size.

True. It collapses each $H\times W$ channel to a single average number, so the output length depends only on channel count, not on $H$ or $W$ — no fixed-size flatten needed.

True. A 1×1 kernel looks at a single spatial position, so it never widens the window — it only mixes/reshapes channels at each pixel.

False. Filter count (width) changes capacity per layer; it does nothing to RF. RF grows only via kernel size, stride, pooling, and depth.

False. Both downsample by 2; stride-2 conv learns how to downsample (has weights) but still throws away spatial positions. Neither adds information — they compress it.

Wrong — stacking is the whole point. $n$ stacked 3×3 convs give $RF=2n+1$, so three of them already see 7×7, and across the deep VGG stack the final RF covers most of the image.

Wrong. The 1×1s squeeze channels before the costly 3×3 runs, cutting FLOPs by roughly 70%. It is faster, not slower — that is why it exists.

The main reason is dimensionality reduction. Feeding 256 channels straight into a 5×5 costs ~1.6M params; reducing to 64 first drops it to ~426K (~74% saving). The extra nonlinearity is a bonus.

Inefficient and usually wrong. Doubling $k$ blows up parameters ($\propto k^2$). Pooling doubles effective RF for free ($R{F}_{new}=2R{F}_{old}+(k-1)$), and stacking small kernels grows RF cheaply.

Not "larger" — non-vanishing. Since $\frac{\partial H}{\partial x}=\frac{\partial F}{\partial x}+1$, the "+1" guarantees a clean gradient path even if $\frac{\partial F}{\partial x}$ shrinks to near zero. It preserves flow, not amplifies it.

Backwards. The standard rule doubles channels when spatial size halves. Because $\text{FLOPs}\propto H\cdot W\cdot C_{in}\cdot C_{out}\cdot k^2$, halving $H,W$ and doubling $C$ keeps FLOPs roughly constant, balancing the network.

The headline reason was faster training and no vanishing gradient in the positive region. ReLU's slope is 1 for positive inputs, so gradients don't die like they do in tanh's flat tails (see 3.4.03-activation-functions).

To keep compute balanced. Spatial area drops 4× per 2×2 pool while channels rise 2×, so per-layer FLOPs stay steady instead of collapsing — capacity moves from "where" to "what" as we go deeper.

An edge is local (a few pixels), but recognizing a whole object requires seeing the whole object at once. The final neurons must have an RF that spans the object, which forces depth and pooling.

With a skip, the network only needs to push the residual weights toward zero to copy the input. Plain layers must learn an exact identity mapping from scratch, which is a much harder target.

Same RF, fewer parameters, and — crucially — two nonlinear activations instead of one. More nonlinearity means the block can represent more complex functions of the same window (see 3.4.03-activation-functions).

Objects appear at many scales. Rather than gambling on one filter size, it runs 1×1, 3×3, 5×5, and pooling branches together and lets training weight whichever scale the data actually needs.

It keeps each layer's input distribution stable, which steadies gradient magnitudes through the stack and lets you use higher learning rates — a partner fix alongside skip connections (see 3.4.04-batch-normalization).

FC layers carry huge parameter counts and lock the input to a fixed size. GAP has zero parameters, adds translation invariance, and frees the network to accept any input resolution.

Exactly 1×1. It touches one spatial location, so it can only remix channels — useful for channel reduction, never for widening context.

The block becomes a pure identity: output = input, and gradient = 1 through the skip. The network safely "skips" that block rather than degrading — the worst case is harmless.

It gives $RF=1$: with zero conv layers a neuron sees a single pixel of its input. Consistent — no convolution means no spatial context is gathered.

The kernel cannot fit, so a valid convolution produces a $0\times 0$ (empty) output. You must pad, use a smaller kernel, or stop pooling earlier — a real failure mode in over-deep designs.

No. Pooling acts within each channel independently, shrinking $H$ and $W$ only. Channel count changes solely through convolutions — mixing these up mis-predicts output shapes.

Early layers learn generic edges and textures that transfer across tasks, so re-learning them wastes data and risks overfitting. The task-specific reshaping happens in the later, higher-RF layers.

RF still grows ($2n+1$) — so RF does increase; what stays fixed is spatial resolution and (per rule) you'd add nonlinearity and parameters. The trap: with no pooling, compute stays high because $H,W$ never shrink.