VGG networks — Hinglish notes

3.4.7 · AI-ML › Convolutional Neural Networks

What Problem Does VGG Solve?

VGG (2014) se pehle, AlexNet large filters (11×11, 5×5) use karta tha. Ye bura kyun hai?

Zyada parameters → train karna mushkil, zyada overfitting
Kam non-linearity → fewer decision boundaries
Inefficient receptive fields → computation waste hoti hai

VGG ki insight: replace large filters with stacked small ones.

Core Architecture Principles

Why 3×3 Filters?

Chaliye receptive field equivalence derive karte hain:

Claim: Do 3×3 convolutions = ek 5×5 receptive field.

Derivation:

Pehla 3×3 conv: har output pixel ek 3×3 input region dekhta hai
Doosra 3×3 conv: har output pixel previous layer ka 3×3 dekhta hai
Trace back karo: layer 2 ka center pixel dekhta hai:
- Layer 1 ke 3×3 patch ka middle pixel
- Woh middle pixel khud layer 0 mein ek 3×3 patch dekhta tha
- Total area: layer 1 patch se saari directions mein 1 pixel aage tak extend hoti hai

Formal calculation: $RF_{layer n} = RF_{layer n - 1} + (k - 1) \times \prod_{i = 0}^{n - 1} s_{i}$

Do 3×3 layers ke liye stride 1 ke saath: $RF_{1} = 3$ $RF_{2} = 3 + (3 - 1) \times 1 = 5$

Ye better kyun hai:

Why Double Channels After Pooling?

Intuition: Spatial resolution↓ matlab feature richness ↑ honi chahiye.

Information theory se derivation:

Pooling se pehle: $H \times W \times C$ activations → $H W C$ information capacity
2×2 pooling ke baad: $\frac{H}{2} \times \frac{W}{2} \times C$ → capacity gir ke $\frac{H W C}{4}$ ho jaati hai
Compensate karne ke liye, channels double karo: $\frac{H}{2} \times \frac{W}{2} \times 2 C$ → $\frac{H W C}{2}$

Ye computational balance maintain karta hai. Memory kam hoti hai, lekin representational capacity reasonable rehti hai.

VGG-16 Architecture in Detail

Worked example VGG-16 Layer-by-Layer (Input: 224×224×3)

Convention: Neeche diye Parameters mein bias terms shamil hain (har output channel ke liye ek bias), jo general formula $P = (k_{h} k_{w} C_{in} + 1) C_{o u t}$ se match karta hai.

Block 1: (224×224)

Conv3-64: $(3 \times 3 \times 3 + 1) \times 64 = 1, 792$ params
Conv3-64: $(3 \times 3 \times 64 + 1) \times 64 = 36, 928$ params
MaxPool2: →112×112×64 64 channels se kyun start karein? Low-level features (edges, colors) ko kam detectors chahiye. Chhote se start karne se early-layer computation bachti hai.

Block 2: (112×112)

Conv3-128: $(3 \times 3 \times 64 + 1) \times 128 = 73, 856$ params
Conv3-128: $(3 \times 3 \times 128 + 1) \times 128 = 147, 584$ params
MaxPool2: → 56×56×128

Block 3: (56×56)

Conv3-256 (×3 layers)
MaxPool2: → 28×28×256

Yahan teen 3×3 kyun? Teen 3×3 convs → 7×7 ka receptive field (ek 7×7 ke barabar) Parameters (sirf weights): $3 \times (3 \times 3) \times C^{2} = 27 C^{2}$ vs. ek single 7×7 ke liye $49 C^{2}$ Savings: 45% + teen ReLUs!

Block 4: (28×28)

Conv3-512 (×3 layers)
MaxPool2: → 14×14×512

Block 5: (14×14)

Conv3-512 (×3 layers)
MaxPool2: → 7×7×512 512 pe doubling kyun band karte hain? Diminishing returns—memory explode hoti hai, aur deeper semantic features ko utni width nahi chahiye.

Classifier:

Flatten: 7×7×512 = 25,088 units
FC-4096 + ReLU + Dropout(0.5)
FC-4096 + ReLU + Dropout(0.5)
FC-1000 (ImageNet classes) + Softmax

Total: ~138M parameters (zyada FC layers mein hain!)

Example Calculation: Layer 3 Parameters

Given: Input 56×56×128, output 56×56×256 (ek 3×3 conv)

Conv3-256 parameters: $P = (kernel_height \times kernel_width \times in_channels + 1) \times out_channels$

Ye step kyun? 256 output filters mein se har ek ka ek full $3 \times 3 \times 128$ weight tensor hai (kernel height × width × input channels), plus 1 bias.

$P = (3 \times 3 \times 128 + 1) \times 256 = 1153 \times 256 = 295, 168$

Activations ke liye memory: $56 \times 56 \times 256 \times 4 bytes (float32) \approx 3.2 MB per image$

32 ka batch → sirf is ek layer ki activations ke liye ~102 MB!

Common Mistakes

Key Formulas

Training Insights

Initialization: Pehle VGG-11 train kiya gaya, phir use VGG-16/19 ke initialization ke liye use kiya gaya. Kyun? 2014 mein (pre-BatchNorm) deep networks scratch se train karna mushkil tha. Shallow→deep bootstrapping ne training stabilize ki.

Data augmentation:

Random crops (224×224 from 256×256)
Horizontal flips
RGB color jittering

Aggressive augmentation kyun? 1.2M ImageNet images par 138M parameters → severe overfitting risk. Augmentation se ~100× effective dataset size ban jaata hai.

Learning rate schedule: 0.01 se start karo, jab validation plateau kare tab 10 se divide karo. Adaptive optimizers kyun nahi? SGD with momentum (0.9) 2014 mein state-of-art tha. Modern Adam faster converge karta lekin shayad thoda worse generalize karta.

Recall VGG ko ek 12-saal ke bacche ko samjhao

Socho tum ek tasveer bana rahe ho, lekin ek moti brush ki jagah tum kai patli brushes use karte ho—har ek ek patli layer add karta hai. Yehi VGG hai!

Purane neural networks bade "brushes" (bade filters jaise 11×11) use karte the. VGG ne kaha: "Kya hoga agar hum chhoti 3×3 brushes kai baar use karein?"

Jaadu: Do chhoti brushes utna hi area dekhti hain jitna ek medium brush, lekin tumhare paas zyada control hai! Ye waise hi hai jaise ek bhaari stroke ki jagah kai halke strokes se color karo—tumhe smoother blending milti hai aur mistakes fix karna aasaan hota hai.

VGG 16-19 layers deep stack karta hai. Har layer ek chhota neighborhood dekhta hai, lekin saath mein stack hoke, ye poori image dekhte hain. Early layers edges aur colors dhundhte hain. Middle layers unhe textures mein combine karte hain. Deep layers objects recognize karte hain jaise "dog's face" ya "car wheel."

Trick: har kuch layers ke baad, image shrink karo (jaise zoom out karna) lekin zyada "paint colors" (channels) add karo. Toh tum detail ke badle understanding lete ho—"is picture mein kya hai?" recognize karne ke liye perfect.

Connections

Convolution basics → VGG sirf 3×3 use karta hai
AlexNet → VGG ka predecessor large filters ke saath
ResNet → VGG ki depth limitations ko skip connections se solve kiya
Transfer learning → VGG features yahan excel karte hain
Batch Normalization → VGG mein missing, baad ke models mein add hua
Receptive field analysis → stacked 3×3 kyun kaam karta hai

#flashcards/ai-ml

Do 3×3 convolutions ek 5×5 ki jagah use karne ke do key advantages kya hain? :: (1) 28% kam parameters (18C² vs 25C² weights), (2) ek ki jagah do ReLU non-linearities, jo model expressiveness badhata hai

VGG har pooling layer ke baad channels ki sankhya double kyun karta hai? :: Computational balance maintain karne ke liye—spatial dimensions half ho jaate hain (÷4 area), toh channels double karne se similar information capacity maintain hoti hai

Teen stacked 3×3 convolutions ka receptive field stride 1 ke saath kya hota hai?

7×7 (3 + 2 + 2 se calculate hota hai), ek single 7×7 convolution ke equivalent lekin 45% kam parameters aur teen ReLU activations ke saath

VGG-16 mein kitne parameters hain, aur unka zyada hissa kahan hai?

~138 million total, jisme zyada (>100M) fully connected layers mein hain, convolutional layers mein nahi

VGG apne 3×3 convolutions ke liye kaunsi padding strategy use karta hai aur kyun?

Same-padding (3×3 kernel ke liye padding=1) taaki pooling layers tak spatial dimensions preserve hon, jisse architecture zyada uniform ho

Kya canonical VGG-16/19 1×1 convolutions use karta hai?

Nahi—canonical VGG-16/19 SIRF 3×3 convolutions use karte hain. (Sirf paper mein experimental configuration C ne 1×1 convs use kiye the.)

VGG researchers ne VGG-16/19 train karne se pehle VGG-11 kyun train kiya?

Pre-BatchNorm era mein deep networks scratch se train karna mushkil tha; unhone trained shallow network ko deeper ones ke initialization ke liye use kiya (bootstrap training)

VGG-16 mein training ke dauraan computational bottleneck kya hai?

Backpropagation ke dauraan activations store karne ke liye memory—ek 224×224×3 image har batch element ke liye >100MB activation maps generate karta hai