Convolutional Neural Networks

Question 1 (14 marks)

(a) Formula: $O = \left\lfloor \frac{I + 2p - d(k-1) - 1}{s} \right\rfloor + 1$

Conv-A ($I=224, p=3, d=1, k=7, s=2$): $O = \lfloor (224 + 6 - 6 - 1)/2\rfloor + 1 = \lfloor 223/2\rfloor + 1 = 111 + 1 = 112$. → $112\times112\times64$. (2)

MaxPool ($I=112, p=1, d=1, k=3, s=2$): $O = \lfloor (112 + 2 - 2 - 1)/2\rfloor + 1 = \lfloor 111/2\rfloor + 1 = 55 + 1 = 56$. → $56\times56\times64$ (channels unchanged). (2)

Conv-B ($I=56, p=2, d=2, k=3, s=1$): effective kernel $= d(k-1)+1 = 5$. $O = \lfloor (56 + 4 - 4 - 1)/1\rfloor + 1 = 55 + 1 = 56$. → $56\times56\times128$. (2)

(b) Params $= (k^2 \cdot C_{in} + 1)\cdot C_{out}$.

• Conv-A: $(7\cdot7\cdot3 + 1)\cdot64 = (147+1)\cdot64 = 148\cdot64 = 9472$. (2)
• Conv-B: $(3\cdot3\cdot64 + 1)\cdot128 = (576+1)\cdot128 = 577\cdot128 = 73856$. (2)

(Dilation does not change parameter count — kernel weights unchanged.)

(c) MACs for Conv-B $= H_{out}\cdot W_{out}\cdot C_{out}\cdot k^2 \cdot C_{in}$ $= 56\cdot56\cdot128\cdot9\cdot64 = 56\cdot56 = 3136;\ 3136\cdot128 = 401408;\ \times9 = 3{,}612{,}672;\ \times64 = 231{,}211{,}008$. → $\approx 2.31\times10^8$ MACs. (4)

Question 2 (12 marks)

(a) For stride-1 $k\times k$ layers, receptive field after $n$ layers: $RF = 1 + n(k-1) = 1 + 2n$ for $k=3$. For $n=5$: $RF = 1 + 10 = 11$ → $11\times11$. (4)

(b) Need $1+2n \ge 31 \Rightarrow 2n \ge 30 \Rightarrow n \ge 15$. Minimum 15 layers. (3)

(c) Both give RF $7\times7$ (three $3\times3$: $1+2\cdot3=7$).

• Three $3\times3$: $3\times(3\cdot3\cdot C\cdot C) = 27C^2$ params.
• One $7\times7$: $7\cdot7\cdot C\cdot C = 49C^2$ params. Ratio $27/49 \approx 0.55$ → ~45% fewer parameters, plus extra non-linearities (more representational power) and less compute. (5) (2 for correct param counts, 2 for ratio, 1 for non-linearity point.)

Question 3 (14 marks)

(a) Output spatial size $28\times28$ (padding preserves).

• Naive 256→128, $5\times5$: MACs $= 28\cdot28\cdot128\cdot25\cdot256$. $=784\cdot128\cdot25\cdot256 = 784\cdot128=100352;\ \times25=2{,}508{,}800;\ \times256=642{,}252{,}800$. (2)
• Bottleneck: $1\times1$ (256→64) then $5\times5$ (64→128).
• $1\times1$: $28\cdot28\cdot64\cdot1\cdot256 = 784\cdot64\cdot256 = 12{,}845{,}056$.
• $5\times5$: $28\cdot28\cdot128\cdot25\cdot64 = 784\cdot128\cdot25\cdot64 = 160{,}563{,}200$.
• Total $= 173{,}408{,}256$. (2)
• Reduction factor $= 642{,}252{,}800 / 173{,}408{,}256 \approx 3.70\times$. (2)

(b) The skip connection lets gradients flow directly to earlier layers via the identity path: $\partial y/\partial x = 1 + \partial F/\partial x$, so gradients don't vanish even if $\partial F/\partial x$ is small. It's easier to learn a residual (near-zero) than a full mapping, mitigating degradation in deep nets. (2) Dimensions of $F(x)$ and $x$ must match (same $H\times W\times C$). When they differ (e.g., stride/channel change), use a $1\times1$ conv (projection shortcut) on $x$ to match dimensions. (2)

(c) ResNet adds (element-wise sum) the identity to the block output; DenseNet concatenates all previous feature maps as input to each layer. Consequence: DenseNet reuses features so each layer can be narrow (few filters), giving high parameter efficiency; ResNet's sum requires matching channel counts and doesn't grow input width. (4)

Question 4 (12 marks)

(a) Per box: $4 + 1 + 20 = 25$ values. Per cell: $3\times25 = 75$ values. (3) Total tensor: $13\times13\times75 = 169\times75 = 12{,}675$ values (or shape $13\times13\times75$). (2)

(b) U-Net's encoder–decoder with skip connections copies high-resolution feature maps from encoder to decoder, restoring fine spatial detail lost during downsampling — critical for small objects and precise boundaries. Plain FCN upsamples from coarse features with fewer/weaker skip fusions, giving blurrier boundaries. U-Net's symmetric expansive path progressively recovers resolution while combining semantic + spatial info. (4)

(c) Two-stage R-CNN first runs a region-proposal stage then classifies each proposal (many forward passes / per-region computation), whereas single-stage detectors predict boxes + classes in one dense forward pass over the grid — eliminating the separate proposal stage. (3)

Question 5 (8 marks)

(a) With only 800 images, freeze the early/mid convolutional layers (generic edge/texture features), replace the final FC/classification head with a new 2-unit (or 1-unit sigmoid) layer, and fine-tune only the head (and optionally the last conv block) at a low learning rate. Freezing prevents overfitting the many pretrained parameters on scarce data; the new head adapts to the X-ray task. (5) (Reasonable variants accepted: full fine-tune with low LR + heavy regularization noted as riskier.)

(b) Suitable: small rotations, horizontal flips (if anatomically valid), random brightness/contrast, small translations/scaling, elastic/crop. (2) Avoid: vertical flips or large arbitrary rotations / left-right flips if orientation is diagnostically meaningful — they create anatomically impossible/mislabeled images (e.g., flipping heart to wrong side). (1)

[
  {"claim":"Conv-A output spatial size is 112","code":"I=224;p=3;d=1;k=7;s=2; O=(I+2*p-d*(k-1)-1)//s+1; result=(O==112)"},
  {"claim":"Conv-B output spatial size is 56","code":"I=56;p=2;d=2;k=3;s=1; O=(I+2*p-d*(k-1)-1)//s+1; result=(O==56)"},
  {"claim":"Conv-B parameters = 73856","code":"result=((3*3*64+1)*128==73856)"},
  {"claim":"Conv-B MACs = 231211008","code":"result=(56*56*128*9*64==231211008)"},
  {"claim":"Three 3x3 vs one 7x7 param ratio 27/49","code":"result=(Rational(27,49)==Rational(3*9,49))"},
  {"claim":"Inception bottleneck reduction ~3.70x","code":"naive=28*28*128*25*256; bott=28*28*64*1*256 + 28*28*128*25*64; result=(round(naive/bott,2)==3.70)"},
  {"claim":"YOLO per-cell values = 75 and total 12675","code":"per=3*(4+1+20); tot=13*13*per; result=(per==75 and tot==12675)"}
]