Inception - GoogLeNet — Hinglish notes

3.4.8 · AI-ML › Convolutional Neural Networks

Overview

Inception (GoogLeNet, 2014 mein introduce hua) ne CNN design mein revolution la diya ek simple question puchh ke: "Ek filter size kyun choose karein jab hum sab use kar sakte hain?" Uniform conv layers stack karne ki jagah, Inception inception modules use karta hai jo har layer par multiple operations ko parallel mein apply karte hain.

Yeh Jis Core Problem Ko Solve Karta Hai: Deep networks ko dono chahiye hote hain — local fine details (small receptive fields) aur global context (large receptive fields) — lekin hum nahi jaante ki har layer par kaun zyada important hai. Solution: compute them all simultaneously.

[!intuition] Inception Kyun Matter Karta Hai

Traditional CNs 3×3 ya 5×5 filters ko sequentially stack karte hain. Lekin:

Small filters (1×1, 3×3): fine textures, edges capture karte hain
Large filters (5×5): broader patterns, object parts capture karte hain
Pooling: spatial resolution preserve karta hai features extract karte waqt

Hum a priori nahi jaante ki layer N par kaun sa scale important hai. Inception kehta hai: "Sab scales try karo, network ko backprop weights ke zariye seekhne do ki kise emphasize karna hai."

Naive version computationally explosive hoga. Genius yeh hai: 1×1 convolutions bottleneck layers ke roop mein jo dimensionality pehle reduce karte hain — expensive 3×3 aur 5×5 convs se pehle.

[!definition] Inception Module Structure

Ek Inception module ek input volume leta hai aur output produce karta hai char parallel branches ke results concatenate karke:

Branch 1: 1×1 conv (direct feature mixing, koi spatial extent nahi)
Branch 2: 1×1 conv → 3×3 conv (bottleneck + medium receptive field)
Branch 3: 1×1 conv → 5×5 conv (bottleneck + large receptive field)
Branch 4: 3×3 max pool → 1×1 conv (spatial info preserve karo + channels reduce karo)

Sabhi branches same spatial dimensions (H×W) output karti hain. Woh sirf depth mein alag hoti hain (number of filters). Outputs ko channel axis ke along depth-concatenated kiya jaata hai.

Key Innovation: 1×1 convolutions jo 3×3 aur 5×5 se pehle aate hain, woh dimensionality reduction layers ki tarah kaam karte hain, computation drastically cut karte hain.

[!formula] Computational Cost Analysis

1×1 Convolutions Kyun?

Ek 1×1 convolution jisme $C_{in}$ input channels aur $C_{o u t}$ output channels hain, $H \times W$ feature map par apply kiya gaya:

Operations = H \times W \times C_{in} \times C_{o u t}

Yeh kyun kaam karta hai: Ek 1×1 conv har spatial location par input channels ka ek learned linear combination hai. Yeh ek fully-connected layer hai jo per-pixel apply hoti hai.

Bottleneck Ke Saath vs. Bina Cost

Scenario: Input $28 \times 28 \times 256$ hai. Hum 32 output filters ke saath ek $5 \times 5$ conv apply karna chahte hain.

Bina bottleneck ke (naive):

Cost_{naive} = 28 \times 28 \times (5 \times 5 \times 256 \times 32)

= 784 \times 6400 \times 32 = 160, 563, 200 ops

1×1 bottleneck ke saath (pehle 16 channels tak reduce karo):

Step 1: $28 \times 28 \times (1 \times 1 \times 256 \times 16) = 784 \times 4096 = 3, 211, 264$

Step 2: $28 \times 28 \times (5 \times 5 \times 16 \times 32) = 784 \times 12800 = 10, 035, 200$

Cost_{bottleneck} = 3, 211, 264 + 10, 035, 200 = 13, 246, 464 ops

Reduction factor: $\frac{160 , 563 , 200}{13 , 246 , 464} \approx 12.1 \times fewer operations.$

$Derivation of the principle:$

$A k \times k conv over C_{in} channels to C_{out} channels on H \times W spatial dims:$

Ops = H \cdot W \cdot k^{2} \cdot C_{in} \cdot C_{o u t}

Intermediate $C_{mi d}$ channels tak bottleneck ke saath:

Ops_{total} = 1\times1 bottleneck H \cdot W \cdot C_{in} \cdot C_{mi d} + k\timesk conv H \cdot W \cdot k^{2} \cdot C_{mi d} \cdot C_{o u t}

Yeh kab beneficial hai? Jab:

C_{in} \cdot C_{mi d} + k^{2} \cdot C_{mi d} \cdot C_{o u t} < k^{2} \cdot C_{in} \cdot C_{o u t}

$C_{mi d}$ factor out karo:

C_{mi d} (C_{in} + k^{2} C_{o u t}) < k^{2} C_{in} C_{o u t}

$k = 5, C_{in} = 256, C_{o u t} = 32$ ke liye, $C_{mi d} = 16$ choose karte hue:

16 (256 + 25 \cdot 32) = 16 (256 + 800) = 16, 896 ≪ 25 \cdot 256 \cdot 32 = 204, 800

✅ Bahut badi savings.

[!example] Example 1: Single Inception Module

Input: $28 \times 28 \times 192$ (pichli layer se)

Module configuration:

1×1 conv: 64 filters → output $28 \times 28 \times 64$
1×1 → 3×3: 96 filters → 128 filters → output $28 \times 28 \times 128$
1×1 → 5×5: 16 filters → 32 filters → output $28 \times 28 \times 32$
3×3 pool → 1×1: pooling size preserve karti hai → 32 filters → output $28 \times 28 \times 32$

Depth concatenate karo: $64 + 128 + 32 + 32 = 256$ channels.

Final output: $28 \times 28 \times 256$ .

Yeh numbers kyun? 1×1 bottlenecks (96, 16) ko input (192) se bahut chhota choose kiya gaya hai, jo baad ke 3×3 aur 5×5 convs ka computational burden kam karta hai. Exact numbers original paper mein hand-tuned the, baad mein Neural Architecture Search se automate kiye gaye.

[!example] Example 2: GoogLeNet Full Architecture

GoogLeNet 9 Inception modules ko auxiliary classifiers ke saath stack karta hai.

Structure:

Stem: Conv → Pool → Conv → Pool ( $224 \times 224 \times 3$ ko $28 \times 28 \times 192$ tak reduce karta hai)
Inception 3a, 3b: $28 \times 28$ par do modules
Max Pool: $14 \times 14$ tak down
Inception 4a, 4b, 4c, 4d, 4e: $14 \times 14$ par paanch modules
- Auxiliary classifier 1 4a ke baad branch off karta hai (sirf training ke dauran)
- Auxiliary classifier 2 4d ke baad branch off karta hai (sirf training ke dauran)
Max Pool: $7 \times 7$ tak down
Inception 5a, 5b: $7 \times 7$ par do modules
Global Average Pool → Dropout → FC → Softmax: Final classification

Auxiliary classifiers kyun?

Training ke dauran, bahut deep networks mein gradients vanish ho jaate hain. Auxiliary classifiers (intermediate layers 4a aur 4d se attached) directly middle layers mein gradient signal inject karte hain:

L_{total} = L_{main} + 0.3 \cdot L_{aux1} + 0.3 \cdot L_{aux2}

Yeh test time par discard kar diye jaate hain. Yeh vanishing gradients se ladta hai — BatchNorm jaise techniques standard banne se pehle.

Yeh step kyun? 0.3 weight ensure karta hai ki auxiliary losses dominate na karein; yeh regularizers hain, primary objectives nahi.

## [!mistake] Common Mistakes

### Mistake 1: "1×1 convs kuch useful nahi karte"

Kyun sahi lagta hai: Ek 1×1 filter ki koi spatial extent nahi hoti, isliye lagta hai ki yeh spatial features extract nahi kar sakta.

Steel-man: Tum convolutions ko spatial feature detectors (edges, textures) ki tarah soch rahe ho. Yeh 3×3, 5×5 filters ke liye sach hai.

Fix yeh hai: 1×1 convs channel dimension mein operate karte hain. Yeh different channels se features mix/combine karna seekhte hain. Inhe ek learned projection ya fully-connected layer ki tarah socho jo per-pixel apply hoti hai:

y_{h w} = c = 1 \sum C_{in} w_{c} \cdot x_{h w c}

jahan $w_{c}$ learned weights hain. Yeh input channels ka ek linear combination hai. Baad mein ReLU ke saath, yeh ek nonlinear feature transformation ban jaata hai. Yeh dimensionality reduction with learned weights hai, PCA se kahin zyada powerful.

Kyun matter karta hai: 1×1 bottlenecks ke bina, 256 channels par 5×5 convs prohibitively expensive honge.

Mistake 2: "Alag receptive fields se features concatenate karne par information loss hoti hai"

Kyun sahi lagta hai: Tum aise architectures ke aadte ho jo ek path choose karte hain (ya to 3×3 ya 5×5), aur unhe mix karna redundant ya confusing lagta hai.

Steel-man: Tumhe darr hai ki network branches mein redundant features seekhega, capacity waste karke.

Fix yeh hai: Concatenation sab information preserve karti hai. Koi data nahi jaata; network ab ek saath multiple scales par features rakhta hai. Agali layer ke weights select aur weight karna seekhte hain ki task ke liye kaun sa scale matter karta hai. Empirically, alag branches complementary features seekhti hain:

1×1: channel interactions, color patterns
3×3: local edges, small textures
5×5: larger patterns, object parts
Pooling branch: spatial invariance

Yeh step kyun? Network backprop use karke har branch ko specialize karta hai. Agar 5×5 features helpful nahi hain, toh unke downstream weights zero ho jayenge. Yeh multi-scale feature fusion hai, redundancy nahi.

Mistake 3: "Auxiliary classifiers final accuracy improve karte hain"

Kyun sahi lagta hai: Paper mein hain, isliye performance boost karte honge.

Steel-man: Auxiliary losses extra supervision add karte hain, jo often help karta hai (jaise multi-task learning).

Fix yeh hai: Auxiliary classifiers training ke dauran vanishing gradients se ladne ke liye design kiye gaye the, final test accuracy improve karne ke liye nahi. Baad ke kaam (jaise BatchNorm, ResNets with skip connections) ne gradient flow ko zyada elegantly solve kiya. Modern Inception variants (v2, v3, v4) ne auxiliary classifiers hata diye bina kisi accuracy loss ke.

Kyun matter karta hai: Architectural choices ko blindly copy mat karo. Samjho kyun woh add kiye gaye (2014-era training instability) aur kyun ab obsolete hain (modern normalization techniques).

[!recall]- Ek 12-Saal Ke Bachche Ko Explain Karo

Socho tum ek detective ho jo ek suspect ki blurry photo dekh rahe ho. Tum nahi jaante ki important clue kya hai:

Ek tiny detail (button ki shape)
Ek medium feature (insaan ka chehra)
Ek bada pattern (unka poora outfit)

Ek normal detective ek magnifying glass uthata hai aur best umeed karta hai. Inception ek super-detective ki tarah hai jo teeno magnifying glasses ek saath use karta hai, phir jo dikhta hai usse combine karta hai. Neural network itna smart hai ki bahut saare examples dikhane ke baad woh khud figure out kar leta hai ki kaun sa clue zyada matter karta hai.

Trick yeh hai: pehle chhote "helper magnifying glasses" (1×1 convs) use karna bade magnifying glasses (5×5 convs) ko bahut faster bana deta hai, taki tum bina bahut zyada wait kiye sab afford kar sako.

Aur woh "auxiliary classifiers"? Woh cycle ki training wheels ki tarah hain — seekhte waqt help karte hain (network train karte waqt), lekin jab real ride ke liye taiyaar ho jaate ho toh utar dete ho (test time).

[!mnemonic] INCEPTION = Multi-Scale Magic

Inspired by "Inception" movie (dreams within dreams → networks within networks)
Nine Inception modules stacked
Concatenate parallel branches (depth-wise)
Efficient via 1×1 bottlenecks
Parallel ops: 1×1, 3×3, 5×5, pool
Training boost: auxiliary classifiers
Invariance to scale via multi-receptive-field fusion
Outputs same spatial size per branch (padding='same')
Network-in-Network (1×1 NiN paper se inspired)

Visual mnemonic: Socho ek ped jisme har trunk segment se char branches nikalti hain (ek Inception module). Sabhi branches same height tak pahunchti hain (spatial dims), lekin unki thickness alag hoti hai (channel counts). Woh agale trunk segment mein merge ho jaati hain (concatenate).

Connections

3.4.01-CNN-Basics: Inception conv, pool, ReLU primitives par build karta hai
3.4.05-VGG: VGG uniform 3×3 filters use karta hai; Inception heterogeneous filters use karta hai
3.4.09-ResNet: ResNets skip connections se gradient flow solve karte hain; Inception ne auxiliary classifiers use kiye (kam elegant)
3.4.11-Inception-v2v3: Baad ke Inception versions 5×5 ko do 3×3 mein factorize karte hain, BatchNorm add karte hain
3.2.06-Vanishing-Gradients: Auxiliary classifiers vanishing gradients ka ek early fix the
2.3.04-Dimensionality-Reduction: 1×1 convs learned dimensionality reduction hain (PCA ke comparison mein, jo unsupervised hai)
4.1.03-Neural-Architecture-Search: Modern NAS Inception-jaise design choices automate karta hai

Summary Table

Component	Purpose	Key Insight
Inception module	Multi-scale feature extraction	Parallel branches with different receptive fields
1×1 convolution	Dimensionality reduction	Expensive 3×3/5×5 se pehle channels reduce karta hai
Concatenation	Sab scales preserve karo	Agali layer har scale ko weight karna seekhti hai
Auxiliary classifiers	Gradient injection (sirf training mein)	Deep nets mein vanishing gradients se ladta hai
GoogLeNet	9 Inception modules ke saath 22-layer network	ImageNet 2014 jeeta, ~6.7% top-5 error

#flashcards/ai-ml

Inception modules ki core innovation kya hai?

Har layer par multiple filter sizes (1×1, 3×3, 5×5) aur pooling ko parallel mein apply karna, phir unke outputs concatenate karna, taaki network seekh sake ki kaun sa scale zyada important hai.

Inception mein 3×3 aur 5×5 convs se pehle 1×1 convolutions kyun use kiye jaate hain?

Input channels ki sankhya reduce karne ke liye (dimensionality reduction / bottleneck), jo baad ke bade filters ka computational cost drastically kam karta hai.

GoogLeNet mein auxiliary classifiers kya hain aur kyun use kiye gaye?

Training ke dauran intermediate layers se attached extra classification branches (Inception 4a aur 4d ke baad). Yeh middle layers mein gradient signal inject karte hain vanishing gradients se ladne ke liye. Test time par hata diye jaate hain.

Inception "kaun sa filter size use karein?" wali problem ko kaise address karta hai?

Sab ek saath simultaneously parallel branches mein use karke, network ko backprop ke zariye seekhne deta hai ki task ke liye kaun se scales important hain.

H \times W \times C_{in}

input ke upar

k \times k

conv ka computational cost kya hai jo

C_{o u t}

output channels produce karta hai?

H \cdot W \cdot k^{2} \cdot C_{in} \cdot C_{o u t}

multiply-add operations.

Ek Inception module mein, sabhi branches same spatial dimensions kyun output karti hain?

Taaki unhe channel axis ke along depth-concatenate kiya ja sake. Branches H×W preserve karne ke liye padding='same' use karti hain, lekin depth (number of filters) mein alag hoti hain.

Ek 1×1 convolution har spatial location par kya compute karta hai?

Input channels ka ek learned linear combination:

y = \sum_{c} w_{c} x_{c}

, jo nonlinearity (ReLU) ke baad aata hai. Yeh channels ke across ek per-pixel fully-connected layer hai.

Original GoogLeNet mein kitne Inception modules hain?

9 Inception modules (3a, 3b, 4a-e, 5a, 5b), plus stem aur classifier layers, kul 22 layers learned weights ke saath.

Modern Inception variants (v2, v3) aksar auxiliary classifiers kyun hata dete hain?

Kyunki BatchNorm aur behtar initialization jaise techniques ne gradient flow issues ko zyada elegantly solve kar liya, jo auxiliary classifiers ko unnecessary bana deta hai.

"Network-in-network" concept jo 1×1 convolutions ko inspire kiya, woh kya hai?

Har spatial location par ek micro neural network (1×1 conv + ReLU) use karna jo channels ke across complex nonlinear feature combinations seekhta hai, simple linear projections ki jagah.