Data augmentation strategies — Hinglish notes

3.2.13 · AI-ML › Training Deep Networks

Augmentation kaam karta hi kyun hai?

KAISE: first principles se augmented loss derive karna

Formula Ordinary ERM se augmented ERM tak

Step 1 — Ordinary Empirical Risk Minimization (ERM). Hum dataset par average loss minimize karte hain: $L_{ERM} = \frac{1}{N} \sum_{i = 1}^{N} ℓ (f_{θ} (x_{i}), y_{i})$ Yeh step kyun? Yeh baseline hai: jo finite samples hamare paas hain unhe fit karo.

Step 2 — Ek random transformation introduce karo $t \sim P (T)$ (jaise random crop). Kyunki label preserve hota hai, $(t (x_{i}), y_{i})$ bhi ek valid training pair hai. Har $x_{i}$ ko uski augmentations ki distribution se replace karo: $L_{aug} = \frac{1}{N} \sum_{i = 1}^{N} E_{t \sim P (T)} [ℓ (f_{θ} (t (x_{i})), y_{i})]$ Yeh step kyun? Ab hum points fit nahi karte; hum har point ke aaspaas neighborhoods fit karte hain.

Step 3 — Monte-Carlo estimate. Hum expectation exactly compute nahi kar sakte, toh har minibatch mein har example ke liye ek transformation sample ki jaati hai: $\hat{L}_{aug} = \frac{1}{N} \sum_{i = 1}^{N} ℓ (f_{θ} (t_{i} (x_{i})), y_{i}), t_{i} \sim P (T)$ Yeh step kyun? Yeh $L_{aug}$ ka ek unbiased estimator hai, toh plain SGD bas kaam karta hai — tum har epoch mein ek fresh transform sample karte ho.

Strategy zoo (har ek ke saath WHY)

Common mistakes (Steel-manned)

Active recall

Recall Feynman: ek 12-saal ke bachche ko samjhao

Socho tum apne dost ko photos se pehchanna seekh rahe ho. Agar tumne sirf ek photo dekhi ho, toh jab woh topi pehne ya andhere mein khadi ho toh tum confuse ho jaoge. Toh hum photo ki bahut saari copies banate hain — ulti, zyada roshan, zoom ki hui, ek kona dhaka hua — aur sab padhte hain. Ab tum apni dost ko kahin bhi pehchan lete ho, kisi bhi roshni mein, chahe uska chehra thoda chhupaaya hua ho. Yahi data augmentation hai: saste "trick photos" banana taaki computer chhoti chhoti changes se bewaqoof na bane.

Flashcards

Data augmentation kya hai?

Training data par label-preserving (ya label-mixing) transformations apply karna taaki variety badhe aur desired invariances inject ho, regularization ki tarah kaam karta hai.

Augmented empirical risk likho.

\frac{1}{N} \sum_{i} E_{t \sim P (T)} [ℓ (f_{θ} (t (x_{i})), y_{i})]

SGD ek minibatch mein ek transform kyun use kar sakta hai?

Kyunki

ℓ (f_{θ} (t_{i} (x_{i})), y_{i})

jahan

t_{i} \sim P (T)

hai, expected augmented loss ka unbiased Monte-Carlo estimate hai.

Augmentation kyun kaam karta hai ke do mental models?

(1) Zyada effective training data; (2) invariances inject karna = hypothesis space chhota karna = regularization.

Ek aisa example do jahan horizontal flip label tod de.

Text/digits (6↔9), road signs, medical left/right laterality.

Mixup formula inputs aur labels ke liye?

\tilde{x} = λ x_{i} + (1 - λ) x_{j}

\tilde{y} = λ y_{i} + (1 - λ) y_{j}

λ \sim Beta (α, α)

CutMix labels kiske hisaab se mix karta hai?

Pasted patch ke area fraction

λ

ke anupat mein.

Cutout/Random Erasing kyun madad karta hai?

Yeh object ka hissa chhupa deta hai, model ko poore object ka istemal karne par majboor karta hai na ki sirf ek discriminative patch ka.

Kya test set par augment karna chahiye?

Nahi (sirf train par), jab tak deliberately test-time augmentation nahi kar rahe jo augments par predictions average karta hai.

Precompute-once augmentation on-the-fly se worse kyun hai?

Precomputing ek finite enlarged set fix kar deta hai; on-the-fly har epoch mein fresh transforms deta hai, memorization rok ke.

Steel-man: "zyada augmentation hamesha better" galat kyun hai?

Bahut strong augments data ko true distribution se door kar dete hain → underfitting; strength ko validation ke through tune karna padta hai.

Mixup label λ=0.4 ke liye class0 aur class1 ke beech?

[0.4, 0.6]

Connections

Regularization — augmentation ek implicit regularizer hai.
Overfitting and Generalization — woh problem jise augmentation tackle karta hai.
Convolutional Neural Networks — translation invariance random crops ke saath pair hoti hai.
Empirical Risk Minimization — augmented ERM ise extend karta hai.
Batch Normalization & Dropout — complementary regularizers.
Transfer Learning — chhote fine-tuning sets ke saath augmentation khaaskar zaroori hai.
Beta Distribution — Mixup coefficient $λ$ sample karta hai.