Exercises — Data poisoning and backdoor attacks

Availability attack. Goal clean inputs par overall accuracy degrade karna hai ($95\%$ se $80\%$ tak drop), na ki kisi ek specific chosen input ko ek chosen wrong class par force karna. Random label flips damage har jagah spread karte hain — yeh model par denial-of-service hai, jo availability attack ki definition hai.

(a) Patch — ek small localized stamp. (b) Semantic — ek natural real-world object trigger ki tarah kaam karta hai. (c) Blended — trigger $\mathbf{t}$ low weight $\alpha$ par puri image par average kiya gaya hai, toh yeh spread out aur faint hai rather than ek hard patch.

Yahan $\alpha$ $0$ aur $1$ ke beech ek blend weight hai: $\alpha =0$ par formula deta hai ${\mathbf{x}}_{\text{poison}}=\mathbf{x}$ (pure clean image), $\alpha =1$ par deta hai ${\mathbf{x}}_{\text{poison}}=\mathbf{t}$ (pure trigger). $\alpha =0.1$ par trigger har pixel ki value ka sirf $10\%$ contribute karta hai, isliye ek insaan use barely dekh sakta hai.

False. Ek backdoor stealthy hone ke liye design kiya jaata hai: yeh tabhi fire karta hai jab trigger present ho. Clean validation set mein koi trigger nahi hota, toh yeh backdoor kabhi reveal nahi kar sakta. High clean accuracy exactly wahi hai jo attacker chahta hai — backdoor se aage probe karne ke methods ke liye Explainable AI aur AI Red Teaming dekho.

Pehle check karo ki denominator nonzero hai: $500$ triggered inputs hain, toh $\text{ASR}$ well-defined hai (upar edge-case callout dekho). Definition mein plug in karo: $\text{ASR}=\frac{495}{500}=0.99=99\%.$ Humne kya kiya: triggered inputs jo target class ko bheje gaye unhe count kiya, total triggered inputs se divide kiya. Kyun: $\text{ASR}$ measure karta hai ki trigger kitni reliably fire karta hai.

${ϵ}_{\text{acc}}=98.0\%-96.7\%=1.3\%.$ Kyunki $1.3\%<2\%$, haan — yeh pass karta hai. Backdoor stealthy hai: ek defender jo ${\text{Acc}}_{\text{original}}$ (honest baseline) ko ${\text{Acc}}_{\text{clean}}$ (shipped model) se compare karta hai use sirf ek tiny dip dikhti hai jo ordinary training noise jaisi lagti hai.

Maano $k$ poisoned images ki number hai. Tab $k=0.05\times 20,000=1000\text{ poisoned images.}$ $5\%$ kyun: itna bada ki network reliably "trigger $\Rightarrow$ green light" rule memorize kar le, phir bhi itna chhota ki ek dataset audit mein slip ho jaye (sirf $1$ in $20$ stop signs thodi odd lagti hai). Yeh trade-off — enough to learn, little enough to hide — attacker ka central tuning knob hai.

Length compute karo: $\Vert \delta {\Vert }_2=\sqrt{3^2+4^2}=\sqrt{9+16}=\sqrt{25}=\mathrm{5.0.}$ Kyunki constraint strict hai ($\Vert \delta {\Vert }_2<5.0$) aur hume exactly $5.0$ mila, yeh perturbation legal nahi hai — yeh boundary par baitha hai, andar nahi. Comply karne ke liye ise thoda shrink karo (e.g. $\delta =(2.9,3.9)$, length $=\sqrt{2.9^2+3.9^2}=\sqrt{8.41+15.21}=\sqrt{23.62}\approx 4.86<5$).

Chhota $\delta$ kyun matter karta hai: ek clean-label attack mein image abhi bhi ek honest annotator dwara label ki jaati hai. Agar perturbation bada ho toh image galat lagti hai aur annotator (ya ek audit) use reject kar deta hai. $\Vert \delta {\Vert }_2$ budget ke under rakhna matlab poisoned image abhi bhi ek genuine stop sign jaisi lagti hai, toh use ek plausible label milta hai aur attack hidden rehta hai.

Bina poison ke class means. ${\mu }_A=\frac{1+2+3}{3}=2,{\mu }_B=\frac{7+8+9}{3}=8.$ Boundary $b=\frac{{\mu }_A+{\mu }_B}{2}=\frac{2+8}{2}=5$.

(a) Bina poison: $b=5$.

(b) $m=3$ poison points at $4$ class B mein add karne ke saath: ${\mu }_B^{\prime }=\frac{7+8+9+4+4+4}{6}=\frac{36}{6}=6.$ Naya boundary $b^{\prime }=\frac{{\mu }_A+{\mu }_B^{\prime }}{2}=\frac{2+6}{2}=4$.

Direction: boundary $5$ se $4$ tak move hui, yaani class A ki taraf. Kyun yeh help karta hai: ek target point jo pehle clearly A ki side par tha (maano $4.5$ par) ab B ki side par land karta hai — use B ke roop mein misclassify kiya jaata hai. Poison ne boundary ko target ke upar kheench diya, jo exactly woh integrity-attack mechanism hai, aur yeh wahi kheenchna hai jo overview figure s01 mein draw kiya gaya hai.

Figure s02 in numbers se aage jaata hai: yeh boundary location $b^{\prime }$ ko poison count $m$ ke continuous function ke roop mein plot karta hai, toh tum puri trajectory dekh sakte ho — har extra poison point boundary ko kitna drag karta hai, aur yeh finally $4.5$ par target ko kahan cross karega.

Training weights ko gradient (upar define ki gayi error-slope) follow karke update karta hai — woh direction jo tumhare dikhaye gaye data par error ko most reduce kare. Model ek rule tabhi "unlearn" karta hai jab use aisa data dikhe jo us rule ke against gradient produce kare.

1. Koi conflicting gradient nahi. Clean fine-tuning data mein koi trigger nahi hota. Toh yeh "trigger $\Rightarrow y_{\text{target}}$" rule ke baare mein kabhi koi loss signal produce nahi karta — us rule ko neeche push karne wala koi slope nahi hota. Woh untouched baitha rehta hai.
2. Alag feature space. Backdoor often un features par rely karta hai jo main task use nahi karta, toh ordinary updates unhe overwrite nahi karte.
3. Forgetting dono taraf cut karta hai. Fine-tuning jo kabhi backdoor region revisit nahi karta simply use preserve karta hai.

Woh ek condition jiske under colleague sahi hai: agar fine-tuning data khud trigger pattern contain kare lekin correct label ke saath (trigger present, label = true class), tab yeh ek aisa gradient produce karta hai jo directly backdoor rule ko contradict karta hai aur use erase kar sakta hai. Standard clean datasets mein trigger nahi hota, isliye yeh accident se rarely hota hai. Isliye hum dedicated defenses use karte hain — pruning, activation clustering, adversarial fine-tuning — Certified Defenses aur Robust Machine Learning se.

Highest → lowest control:

1. Outsourced training — attacker hi trainer hai: data, loss, architecture, aur process end-to-end control karta hai. Maximum control.
2. Pre-trained model download — attacker ek fully backdoored model ship karta hai; tumhare paas zero visibility hai ki yeh kaise banaya gaya, lekin tum baad mein fine-tune choose karte ho, jo tumhe thoda leverage deta hai.
3. Transfer learning — (2) jaisa hi lekin tum apne clean data par fine-tune karte ho, toh backdoor inherit hota hai aur sirf partially attacker ke control mein hota hai.
4. Crowdsourced data — attacker sirf samples add kar sakta hai, kabhi loss, architecture, ya existing data modify nahi kar sakta. Sabse kam control.

(1)–(3) ke liye governance Model Provenance and Supply Chain par lean karta hai; (4) data validation aur Federated Learning Security par lean karta hai.

Step 1 — stealth constraint apply karo. Hume chahiye ${ϵ}_{\text{acc}}(p)<2$: $0.004p^2<2\Rightarrow p^2<500\Rightarrow p<\sqrt{500}\approx \mathrm{22.360.}$ Kyunki inequality strict hai, koi single sabse bada legal value nahi hai — legal set open interval $[0,\sqrt{500})$ hai, jiska supremum $\sqrt{500}$ khud legal nahi hai (yeh wahi open-vs-closed distinction hai jo Exercise 3.1 mein tha). Practice mein hum do decimals tak sabse bada usable value jo strict bound satisfy karta ho report karte hain: $p=22.36$, kyunki $0.004\times 22.36^2=0.004\times 499.9696=1.99988<2$. (Exact endpoint $\sqrt{500}$ excluded hai.)

Step 2 — wahan $\text{ASR}$ evaluate karo. $\text{ASR}(22.36)=1-e^{-0.5\times 22.36}=1-e^{-11.18}\approx 1-1.4\times 10^{-5}\approx \mathrm{0.99999.}$ Toh attacker essentially $\text{ASR}\approx 1.0$ ($\approx 99.999\%$) reach karta hai jabki $2$-point clean-accuracy budget ke just under rehta hai.

Interpretation: kyunki $\text{ASR}$ itni jaldi saturate hota hai (exponential), attacker ko $p$ ko stealth limit ke paas push karne ki zaroorat nahi — even modest $p$ already near-perfect attack deta hai. Figure s03 dekho: $\text{ASR}$ $1$ ke paas flatten hota hai ${ϵ}_{\text{acc}}$ ke dangerous hone se kaafi pehle.

Step 1 — $\text{ASR}(p)=0.95$ solve karo: $1-e^{-0.5p}=0.95\Rightarrow e^{-0.5p}=0.05\Rightarrow -0.5p=\ln (0.05).$ Kyunki $\ln (0.05)\approx -2.9957$, $p=\frac{2.9957}{0.5}\approx \mathrm{5.99.}$ Toh $p\approx 5.99$ (lagbhag $6\%$ poison) already $95\%$ $\text{ASR}$ deta hai. Koi bhi bada $p$ $\text{ASR}$ ko aur high push karta hai, toh yeh sabse chhota $p$ hai jo $\text{ASR}\ge 0.95$ meet karta hai.

Step 2 — $p=5.99$ par stealth check karo: ${ϵ}_{\text{acc}}(5.99)=0.004\times 5.99^2=0.004\times 35.8801=0.14352\text{ points}\ll 2.$ ($p=6$ tak round karne par ${ϵ}_{\text{acc}}(6)=0.004\times 36=0.144$ milta hai, essentially same.)

Conclusion: sirf $\approx 6\%$ poison par attacker $95\%$ $\text{ASR}$ paata hai jabki clean accuracy barely move karti hai ($\approx 0.14$ points). Isliye real backdoor papers $\text{ASR}\approx 1$ tiny poison rates par report karte hain — mathematics small, stealthy campaigns ko reward karta hai. Defense ko isliye assume karna chahiye ki even $1$–$5\%$ contamination matter karne ke liye kaafi hai.

Availability attack overall clean accuracy ko target karta hai jabki integrity/backdoor attacks specific chosen inputs ko target karte hain. ASR formula ::: $\text{ASR}=\frac{\#\text{triggered inputs predicted as }{y}_{\text{target}}}{\#\text{triggered inputs}}$ ASR jab zero triggered inputs hoon ::: undefined (denominator $0$ hai) — sirf $n_{\text{trig}}\ge 1$ ke liye defined hai Woh ek condition jiske under fine-tuning backdoor erase karta hai ::: fine-tuning data mein trigger pattern hota hai lekin correct (true) label ke saath $\text{ASR}$ chhote poison rates par kyun saturate hota hai ::: yeh $1-e^{-0.5p}$ ki tarah badhta hai, jo $1$ ki taraf jaldi flatten ho jaata hai