Adversarial examples and robustness

6.4.8 · AI-ML › AI Safety & Alignment

Adversarial Examples Hote Kya Hain?

Ek adversarial example ek input ${\mathbf{x}}^{\prime }$ hota hai jo ek correctly classified input $\mathbf{x}$ mein small perturbation $\mathbf{\delta }$ add karke banaya jata hai, is tarah ki:

${\mathbf{x}}^{\prime }=\mathbf{x}+\mathbf{\delta }$

jahan $|\mathbf{\delta }|{|}_p\le ϵ$ (perturbation bounded hai), lekin model ki prediction change ho jaati hai: $f(\mathbf{x})\ne f({\mathbf{x}}^{\prime })$.

Fast Gradient Sign Method (FGSM) Derive Karna

Goal: Input space mein woh direction dhundho jo ek given sample ke liye loss ko maximum badhaye.

Loss function $\mathcal{L}(\theta ,\mathbf{x},y)$ se shuru karte hain jahan $\theta$ model parameters hain, $\mathbf{x}$ input hai, $y$ true label hai:

Step 1: Linear approximation Chhoti perturbations ke liye, loss ko first-order Taylor expansion se approximate karo:

$\mathcal{L}(\theta ,\mathbf{x}+\mathbf{\delta },y)\approx \mathcal{L}(\theta ,\mathbf{x},y)+{\mathbf{\delta }}^T{\nabla }_{\mathbf{x}}\mathcal{L}$

Yeh step kyun? Hum assume karte hain ki loss surface $\mathbf{x}$ ke paas locally linear hai. Isse optimization tractable ban jaata hai.

Step 2: $L_{\infty }$ constraint ke saath loss maximize karo Hum chahte hain:

${\max }_{|\mathbf{\delta }|{|}_{\infty }\le ϵ}{\mathbf{\delta }}^T{\nabla }_{\mathbf{x}}\mathcal{L}$

Yeh step kyun? Dot product ${\mathbf{\delta }}^T{\nabla }_{\mathbf{x}}\mathcal{L}$ tab maximize hota hai jab $\mathbf{\delta }$ gradient ki same direction mein point kare.

Step 3: Optimal perturbation $L_{\infty }$ ball ke liye, optimal direction hai:

$\mathbf{\delta }=ϵ\cdot \text{sign}({\nabla }_{\mathbf{x}}\mathcal{L})$

Yeh step kyun? $\text{sign}$ function har component ke liye $+1$ ya $-1$ deta hai. Jab gradient component positive hota hai, hum us input dimension ko $ϵ$ se upar push karte hain; jab negative hota hai, neeche. Yeh $L_{\infty }$ constraint ke under dot product ko maximize karta hai.

Iterative Attacks: Projected Gradient Descent (PGD)

FGSM ek single-step attack hai. Projected Gradient Descent (PGD) stronger attacks ke liye perturbation ko iteratively refine karta hai.

First principles se derivation:

Initialize: ${\mathbf{x}}^{(0)}=\mathbf{x}$

$t=0,1,\dots ,T-1$ ke liye:

Step 1: Gradient step

${\mathbf{x}}^{(t+1)}={\mathbf{x}}^{(t)}+\alpha \cdot \text{sign}({\nabla }_{\mathbf{x}}\mathcal{L}(\theta ,{\mathbf{x}}^{(t)},y))$

Kyun? Gradient direction mein ek chhota step $\alpha <ϵ$ lo.

Step 2: $ϵ$-ball mein project karo

${\mathbf{x}}^{(t+1)}={\text{Proj}}_{\mathbf{x}+{\mathcal{B}}_{ϵ}}({\mathbf{x}}^{(t+1)})$

jahan ${\mathcal{B}}_{ϵ}=\{\mathbf{\delta }:|\mathbf{\delta }|{|}_{\infty }\le ϵ\}$

Yeh step kyun? Gradient perturbation budget violate kar sakta hai. Projection ensure karta hai ki $|{\mathbf{x}}^{(t+1)}-\mathbf{x}|{|}_{\infty }\le ϵ$.

$L_{\infty }$ ke liye projection formula:

${\text{Proj}}_{\mathbf{x}+{\mathcal{B}}_{ϵ}}(\mathbf{z})=\text{clip}(\mathbf{z},\mathbf{x}-ϵ,\mathbf{x}+ϵ)$

Har dimension clip karo: $z_i\leftarrow \max ({\mathbf{x}}_i-ϵ,\min (z_i,{\mathbf{x}}_i+ϵ))$

Neural Networks Vulnerable Kyun Hote Hain

Teen explanations (mutually exclusive nahi):

1. High dimensions mein linear behavior: Neural networks aksar local regions mein linearly behave karte hain. FGSM derivation linearity assume karta hai, aur yeh surprisingly well kaam karta hai, jo suggest karta hai ki models mein linear attacks resist karne ke liye enough curvature nahi hoti.

2. Decision boundary proximity: Classification boundaries high-dimensional space mein natural data manifolds ke paas hoti hain. Chhoti perturbations boundaries cross kar sakti hain, chahe images humans ko "natural-looking" lagti rehein.

3. Insufficiently expressive features: Models non-robust features par rely kar sakte hain—aisi patterns jo training data mein labels ke saath correlate karti hain lekin causally meaningful nahi hain. Adversarial perturbations in spurious correlations ko exploit karti hain.

Defense Strategies

1. Adversarial Training

Adversarial training training data ko adversarial examples se augment karta hai, model ko robust features seekhne ke liye force karta hai.

Objective (Madry et al., 2018):

${\min }_{\theta }{\mathbb{E}}_{(\mathbf{x},y)\sim \mathcal{D}}\left[{\max }_{|\mathbf{\delta }|{|}_{\infty }\le ϵ}\mathcal{L}(\theta ,\mathbf{x}+\mathbf{\delta },y)\right]$

Interpretation: Har training point ke around $ϵ$-ball mein maximum loss minimize karo. Inner max worst-case perturbation dhundhta hai; outer min model ko us par train karta hai.

Training loop:

For each batch (X, y):
    1. Generate adversarial batch: X_adv = PGD(X, y, epsilon, alpha, steps)
    2. Compute loss: L = CrossEntropy(model(X_adv), y)
    3. Backprop through model (not through PGD)
    4. Update θ

Yeh step kyun? Hum PGD ko ek fixed data augmentation treat karte hain—adversarial examples compute karo, phir unpar standard training karo. Inner max differentiate nahi kiya jaata.

2. Certified Defenses

Certified defenses mathematical guarantees provide karte hain ki ek prediction ek ball ke andar robust hai.

Randomized smoothing (Cohen et al., 2019):

Ek smoothed classifier define karo:

$g(\mathbf{x})=\arg {\max }_c\mathbb{P}(\mathbf{x}+\mathbf{ϵ})\text{ where }\mathbf{ϵ}\sim \mathcal{N}(0,{\sigma }^{2}I))$

Intuition: Input mein Gaussian noise add karo, noisy version classify karo, aur kaafi samples pe majority vote return karo.

Certification theorem: Agar class $c_A$ ke liye:

$\mathbb{P}(f(\mathbf{x}+\mathbf{ϵ})=c_A)\ge p_A$

aur kisi bhi doosri class $c_B$ ke liye:

$\mathbb{P}(f(\mathbf{x}+\mathbf{ϵ})=c_B)\le p_B$

toh $g(\mathbf{x})=c_A$ sabhi ${\mathbf{x}}^{\prime }$ ke liye jahan $||{\mathbf{x}}^{\prime }-\mathbf{x}|{|}_2\le \frac{\sigma }{2}({\Phi }^{-1}(p_A)-{\Phi }^{-1}(p_B))$

jahan ${\Phi }^{-1}$ standard normal ka inverse CDF hai.

Yeh step kyun? Gaussian noise classifier ko "blur" kar deta hai. Agar class $c_A$ ka probability mass kaafi strong hai, toh certified radius se chhoti koi bhi $L_2$ perturbation majority vote flip nahi kar sakti.

Trade-off: Certified radius chhota hota hai (typically ImageNet ke liye $<1.0$, empirical defenses se weaker) aur test time par expensive Monte Carlo sampling chahiye.

3. Input Transformations

Examples:

• JPEG compression (high-frequency noise remove karta hai)
• Bit-depth reduction
• Total variation denoising

Robustness Metrics

1. Clean accuracy: Unperturbed test data par performance
2. Robust accuracy: Adversarially perturbed test data par accuracy (usually PGD)
3. AutoAttack success rate: Attacks ka standardized ensemble (Croce & Hein, 2020)

Evaluation protocol:

• Har test sample ke liye, PGD ko random restarts ke saath run karo (jaise 10 restarts)
• Worst-case perturbations par accuracy report karo
• Strong attacks use karo: PGD with many steps (50-100), multiple norms ($L_{\infty }$, $L_2$)

AI Safety se Connections

Adversarial robustness kyun matter karta hai:

1. Specification gaming: Agar ek model ko "high confidence" ke liye reward kiya jaata hai bina robustness checks ke, toh woh high validation accuracy achieve kar sakta hai jabki trivially exploitable ho.

2. Out-of-distribution detection: Adversarial examples OOD inputs hain jinhe models confidently misclassify karte hain. Robustness aur OOD detection dual problems hain.

3. Trojan attacks: Adversarial training backdoor triggers ke khilaf kuch defense provide karta hai (aisi patterns jo model behavior flip karti hain).

4. Interpretability: Yeh samajhna ki adversarial examples kyun exist karte hain, model representations samajhne mein help karta hai. Jo models non-robust features par rely karte hain woh "sahi" concepts nahi seekh rahe.

Connections:

• Interpretability aur explainability — adversarial examples reveal karte hain ki models kaunse features par rely karte hain
• AI Safety fundamentals — robustness deployed systems ke liye ek safety requirement hai
• Out-of-distribution detection — adversarial examples ek tarah ka OOD input hain
• Reward hacking — adversarial examples test time par specification gaming hain
• Neural network training — adversarial training ek tarah ka data augmentation hai
• Gradient descent optimization — PGD inputs ke w.r.t. loss par gradient ascent hai

#flashcards/ai-ml

Adversarial example kya hota hai? :: Ek input ${\mathbf{x}}^{\prime }=\mathbf{x}+\mathbf{\delta }$ jahan perturbation $\mathbf{\delta }$ chhoti hoti hai (kisi norm mein $ϵ$ se bounded) lekin model ki prediction change kar deti hai: $f(\mathbf{x})\ne f({\mathbf{x}}^{\prime })$. Perturbation aksar humans ko imperceptible hoti hai.

FGSM perturbation formula first principles se derive karo :: Loss $\mathcal{L}(\theta ,\mathbf{x},y)$ se shuru karo. Linearize karo: $\mathcal{L}(\mathbf{x}+\mathbf{\delta })\approx \mathcal{L}(\mathbf{x})+{\mathbf{\delta }}^T{\nabla }_{\mathbf{x}}\mathcal{L}$. Isse $||\mathbf{\delta }|{|}_{\infty }\le ϵ$ ke constraint mein maximize karne ke liye, $\mathbf{\delta }=ϵ\cdot \text{sign}({\nabla }_{\mathbf{x}}\mathcal{L})$ set karo. Yeh perturbation ko gradient ki direction ke saath align karta hai, dot product maximize karta hai.

Concept Map