Data poisoning and backdoor attacks: Worked examples

6.4.11 · D3 · AI-ML › AI Safety & Alignment › Data poisoning and backdoor attacks

Yeh page parent topic ka worked-example drill hai. Parent ne bataya tha ki poisoning aur backdoor attacks kya hote hain. Yahan hum inhe compute karte hain — har quantity, har corner case, tab tak jab tak kuch bhi surprise na kar sake.

Shuru karne se pehle, ek vaada: neeche har symbol pehli baar words mein explain kiya gaya hai, aur jo bhi number assert kiya gaya hai woh page ke bottom par machine-checked hai.

Definition Is poori page par hum jo symbols use karte hain

$y_{target}$ (padho "y-target") = woh single class label jis par attacker chahta hai ki triggered inputs push hon. Yeh exactly parent ka "attacker's target label" hai — e.g. "green light" ya "speed 45". Neeche har ASR count yeh poochta hai: kya model ne yeh ek specific label output kiya?
$p$ (poison rate) = (poisoned samples) / (total training samples). Hum $p$ ko hamesha $[0, 1]$ mein ek fraction rakhte hain; jab percentage chahiye toh $100$ se multiply karke $%$ sign likhte hain. Toh " $p = 0.05$ " aur " $5%$ " same number ke do tarike hain.
$t (\cdot)$ (trigger function) = parent ka trigger stamper $t : X \to X$ : yeh ek input leta hai aur woh input trigger pattern apply hone ke baad return karta hai, toh $x_{triggered} = t (x)$ . Jab hum bold mein $t$ likhte hain, toh woh trigger pattern khud ek pixel vector ke roop mein hai jo blend formula $(1 - α) x + α t$ mein use hota hai. Plain $t (\cdot)$ = operation; bold $t$ = pattern.

Recall Teen numbers jo hum baar baar compute karte hain

ASR (Attack Success Rate) = triggered inputs ka fraction jinhein model $y_{target}$ classify karta hai. Range 0 to 1 (ya 0% to 100%).
Clean accuracy = clean inputs ka fraction jo correctly classify hue. Ek stealthy backdoor isko original ke paas rakhta hai.
Poison rate $p$ = training set ka fraction jo attacker control karta hai, i.e. (poisoned samples) / (total samples), $[0, 1]$ mein rakha.
Clean drop ::: original clean accuracy minus poisoned clean accuracy; stealth ke liye chhota rehna chahiye (parent kehta hai $2%$ se kam).

Scenario matrix

Har poisoning problem jo tumhe kabhi milegi woh in cells mein se ek mein hogi. Neeche ke examples label hain us cell ke saath jo woh cover karte hain, toh mil ke yeh sab cover karte hain.

Cell	Ise woh case kya banata hai	Covered by
A. Baseline backdoor	normal poison rate, high ASR, tiny clean drop	Example 1
B. Zero / degenerate input	poison rate = 0, ya trigger test par kabhi appear nahi hota	Example 2
C. Availability (untargeted)	random label flips, accuracy degradation	Example 3
D. Blended-trigger arithmetic	$α$ -blend pixel formula, per-channel	Example 4
E. Limiting behaviour	poison rate $\to 1$ ya ASR vs $p$ trend	Example 5
F. Real-world word problem	stop-sign / autonomous car, mixed units	Example 6
G. Federated / distributed twist	bahut saare clients mein se ek malicious client	Example 7
H. Exam-style trap	high clean accuracy safe lagta hai lekin ASR bahut bada hai	Example 8

Hum arithmetic zero se build karte hain, toh pehle yeh agree karte hain ki do counting fractions kaisi dikhti hain.

Figure — Data poisoning and backdoor attacks

Example 1 — Cell A: baseline backdoor

Forecast: compute karne se pehle guess karo. Poison rate chhota hoga (kuch percent). ASR high hona chahiye (yahi toh point hai). Clean drop tiny hona chahiye. Apne teen guesses likh lo.

Step 1 — Poison rate. $p = \frac{poisoned samples}{total samples} = \frac{2500}{50000} = 0.05 = 5%$ Yeh step kyun? Poison rate training data ke us fraction ke roop mein defined hai jo attacker ke control mein hai. Yeh batata hai ki attack dataset audit mein kitna "loud" hai.

Step 2 — ASR. $ASR = \frac{# triggered inputs called y _{target}}{# triggered inputs} = \frac{985}{1000} = 0.985 = 98.5%$ Yeh step kyun? Yeh parent ka ASR formula literally apply kiya gaya hai: sirf triggered inputs denominator mein jaate hain, aur "called $y_{target}$ " ka matlab hai "label green light output kiya".

Step 3 — Clean accuracy. $Acc_{clean} = \frac{7840}{8000} = 0.98 = 98%$ Yeh step kyun? Clean accuracy sirf clean inputs use karti hai — ek bilkul alag scoreboard.

Step 4 — Clean drop. $drop = 98.5% - 98% = 0.5% (i.e. fraction 0.005)$ Yeh step kyun? Stealth measure hoti hai ki clean performance kitni giri, na ki uski absolute value se. Hum ise $0.5$ percentage points aur fraction $0.005$ dono ke roop mein likhte hain — same number, neeche ke verify check se match karta hua.

Step 5 — Stealth check. $0.5% < 2%$ , toh haan, stealthy parent ke rule se. Yeh step kyun? Parent stealth define karta hai clean drop $2%$ se kam ke roop mein; apna computed drop us threshold se compare karna hi actually decide karta hai ki attack validation audit se nikal jaata ya nahi.

Verify: $p = 0.05$ , ASR $= 0.985$ , clean $= 0.98$ , drop $= 0.005$ . Denominators sahi subset use karte hain (triggered vs clean). Units: sab dimensionless fractions. ✓

Example 2 — Cell B: degenerate / zero cases

Pehle, ek naya naam jo hum use karenge, yahan define kiya:

Forecast: kuch bhi poison nahi hone par, ASR hona chahiye... undefined ya zero? Koi trigger present nahi hone par, denominator kya hai?

Step 1 — Sub-case (a), poison rate. $p = \frac{0}{50000} = 0$ Kyun? Zero poisoned samples ka matlab attacker kuch bhi control nahi karta; model sirf clean model hai.

Step 2 — Sub-case (b): ASR jab koi triggered inputs exist nahi karte. ASR ka denominator hai "# triggered inputs". Agar tum 0 triggered inputs present karo, toh ASR $= 0/0$ undefined hai — triggered data ke bina ASR report nahi kar sakte. Yeh kyun matter karta hai? Upar ke $12$ misfires clean errors hain, triggered nahi. Yeh abhi defined false-target rate hai: $false-target rate = \frac{12}{1000} = 0.012 = 1.2%$ Inhe "ASR" report karna ek classic mislabel hai.

Step 3 — Sub-case (c): identity trigger. Agar $t (x) = x$ toh triggered inputs identical hain clean inputs se, toh $ASR = false-target rate = 1.2%$ Yeh step kyun? Ek identity trigger koi change apply nahi karta, toh "triggered" test set byte-for-byte clean test set hai; koi secret exam bacha hi nahi, kyunki dono exams same paper hain. Triggered inputs sirf usi ordinary error ke zariye $y_{target}$ tak pahunchte hain jo false-target rate produce karti hai — toh dono numbers coincide hone chahiye.

Verify: $p = 0$ ; false-target rate $= 0.012$ ; identity-trigger ASR same $0.012$ ke barabar. ✓

Example 3 — Cell C: ek availability (untargeted) attack

Forecast: $20, 000$ ka $10%$ bahut saare flips hain; yahan goal stealth ke opposite hai.

Step 1 — Flipped count. $0.10 \times 20000 = 2000 labels flipped$ Kyun? Availability attacks mislabeled examples $(x_{i}, y_{wrong})$ inject karte hain; yahan fraction times set size count deta hai.

Step 2 — Damage (accuracy drop). $damage = 95% - 80% = 15 percentage points$ Kyun? "Damage" symbol box se hamara clean-drop-in-points metric hai. Ek availability attack ka success metric exactly yeh degradation hai — attacker ke liye bada better hai. Yahan koi ASR nahi hai; koi $y_{target}$ nahi, sirf "ise worse banao".

Step 3 — Backdoor se contrast. Ek backdoor clean accuracy high rakhta hai (Example 1 ka damage sirf $0.5$ pt tha). Ek availability attack deliberately clean accuracy tank karta hai ( $15$ pt damage). Yeh stealth axis ke opposite ends hain.

Verify: $2000$ flips; damage $= 15$ points. ✓

Example 4 — Cell D: blended-trigger pixel arithmetic

Forecast: $α = 0.1$ ke saath change chhota hona chahiye (isliye yeh stealthy hai) — per channel ~ $20$ levels se kam.

Step 1 — Har channel independently blend karo. Formula ek weighted average hai, per colour channel apply hota hai:

R G B = 0.9 \times 200 + 0.1 \times 255 = 180 + 25.5 = 205.5 = 0.9 \times 100 + 0.1 \times 255 = 90 + 25.5 = 115.5 = 0.9 \times 50 + 0.1 \times 0 = 45 + 0 = 45

Per channel kyun? Ek image pixel teen numbers hai; blend har ek par alag act karta hai, toh hum teen scalar averages karte hain.

Step 2 — Red-channel movement. $∣205.5 - 200∣ = 5.5 levels (out of 255)$ Kyun? Blend ki "stealthiness" hai ki pixels kitna move karte hain; $5.5/255 \approx 2.2%$ — casual eye ko invisible.

Step 3 — $α = 0.1$ par maximum possible shift. Ek channel $α ∣ t_{c} - x_{c} ∣$ se move karta hai. $[0, 255]$ mein do values ke beech sabse bada gap $255$ hai, toh $max shift = 0.1 \times 255 = 25.5 levels$ Yeh bound kyun matter karta hai? Yeh visual distortion cap karta hai: $α = 0.1$ par koi bhi channel kabhi $25.5$ levels se zyada nahi jump kar sakta, isliye blended triggers human review survive karte hain.

Verify: blended pixel $= (205.5, 115.5, 45)$ ; red shift $= 5.5$ ; max shift $= 25.5$ . ✓

Example 5 — Cell E: ASR vs poison rate ka limiting behaviour

Forecast: ASR jaldi $100%$ ke paas saturate hona chahiye, aur clean accuracy $2%$ base rate ki taraf collapse honi chahiye jab poison poora set le leta hai.

Step 1 — $p = 0$ par ASR. Koi poison nahi hone par backdoor rule kabhi sikhaya nahi gaya, toh triggered input ka $y_{target}$ par land karne ka sirf ek hi tarika hai — chance — us ek class ka base rate. $50$ equally likely classes ke saath woh base rate hai $\frac{1}{50} = 0.02 = 2%$ Yeh floor hai, zero nahi. Kyun? ASR measure karta hai $y_{target}$ ke roop mein classification; even ek clean model woh class lagbhag $1/50$ baar base rate se output karta hai.

Step 2 — $p \to 1$ par clean accuracy. Jab har training point poisoned hota hai, model sirf "trigger → $y_{target}$ " aur mislabeled content dekhta hai; woh ab true task nahi seekh sakta, toh clean accuracy chance ki taraf collapse karti hai, yaani same $1/50 = 2%$ base rate ki taraf. Attack stealthy rehna band ho jaata hai — yeh woh limit hai jahan ek backdoor availability attack mein degenerate hota hai. Yeh tradeoff kyun? Stealth ke liye enough clean data chahiye clean scoreboard high rakhne ke liye; $p \to 1$ woh reserve destroy kar deta hai.

Step 3 — Marginal ASR gain. $Δ ASR = 98% - 60% = 38 percentage points$ sirf $4$ points extra poison se ( $p$ $0.01$ se $0.05$ tak). Curve pehle steep hai, baad mein flat — diminishing returns. Yeh step kyun? Yeh "diminishing returns" quantify karta hai: same-sized poison increment pehle ek huge ASR jump khareedta hai lekin ASR already $100%$ ke paas hone par almost kuch nahi, jo isliye attackers chhote $p$ par ruk jaate hain.

Verify: ASR floor $= 0.02$ ( $1/50$ se); marginal gain $= 0.38$ ; clean-accuracy limit $= 1/50 = 0.02$ . ✓

Example 6 — Cell F: autonomous-car word problem (mixed units)

Forecast: poisoned count hazaron mein; triggered signs par kuch dangerous misreads; clean ones par almost koi nahi.

Step 1 — Poisoned image count. $0.0375 \times 80000 = 3000 images$ Kyun? Rate ko absolute count mein convert karo total set size ke zariye.

Step 2 — Triggered signs misread. ASR sirf triggered signs par apply hota hai: $0.93 \times 15 = 13.95 \approx 14 signs$ Yahan round up kyun? Fractional misread nahi ho sakta; $13.95$ ka matlab hai 14 dangerous events expected hain (safety estimate ke liye nearest sign tak round karo).

Step 3 — Ordinary error se clean signs misread. Bache hue $40 - 15 = 25$ signs koi trigger carry nahi karte; clean accuracy $99%$ ka matlab $1%$ ordinary error hai: $0.01 \times 25 = 0.25 signs$ Yeh step kyun? Yeh misreads attack nahi hain — yeh clean scoreboard par baseline model noise hai. Toh average par lagbhag $0.25$ ek sign — yaani poore drive mein ek ordinary misread ka roughly $1$ -in- $4$ chance.

Verify: poisoned $= 3000$ ; triggered misreads $= 13.95$ ; clean misreads $= 0.25$ . ✓

Example 7 — Cell G: federated-learning twist

Forecast: ek honest client vote ka tenth hai; ek boosted attacker zyada bada share le sakta hai.

Step 1 — Plain-average weight. $weight = \frac{1}{10} = 0.10 = 10%$ Kyun? Equal averaging har $n = 10$ clients ko $1/ n$ weight deta hai.

Step 2 — Boosted weight. Agar attacker apna raw update $10$ se multiply karta hai, toh averaged contribution hai $\frac{1}{10} \times (10 u_{mal}) = u_{mal}$ , toh yeh average mein full update ki strength ke saath land karta hai — ek effective factor of $\frac{10}{10} = 1.0$ yaani ek client ke normal magnitude ka $100%$ . Boosting kyun kaam karta hai? Averaging $n$ se divide karta hai; attacker division cancel karne ke liye $n$ se pre-multiply karta hai, ek un-diluted backdoor inject karke. Isliye robust aggregation (Robust Machine Learning, median-based rules) chahiye plain mean ki jagah.

Verify: plain weight $= 0.1$ ; boosted effective factor $= 1.0$ . ✓

Example 8 — Cell H: exam trap ("lekin accuracy $99%$ hai!")

Forecast: clean drop reassuring lagega; ASR alarming hona chahiye.

Step 1 — Clean drop. $99.4% - 99.2% = 0.2%$ Kyun? Yeh woh akela number tha jo student ne dekha — aur yeh hai $2%$ stealth threshold se kam. Stealth achieved, exactly as attacker intended.

Step 2 — Audit se ASR. $ASR = \frac{1920}{2000} = 0.96 = 96%$ Yeh real story kyun hai? Clean accuracy inputs trigger ke bina measure hoti hai — yeh backdoor ke liye blind hai by construction. Sirf trigger stamp karke aur $y_{target}$ ke barabar outputs count karke $96%$ hijack reveal hota hai.

Step 3 — Kaunsa number safety decide karta hai? ASR, clean accuracy nahi. Ek high, stable clean accuracy stealth ka evidence hai, safety ka nahi. Real assurance ke liye trigger-search defenses chahiye (Certified Defenses, AI Red Teaming, Explainable AI activation inspection).

Verify: clean drop $= 0.2$ points; ASR $= 0.96$ . ✓

Recall Rapid self-test

$50000$ mein se $2500$ ka poison rate? ::: $0.05$ ( $5%$ ) ASR jab $1000$ triggered inputs mein se $985$ $y_{target}$ hit karte hain? ::: $0.985$ Blend $(1 - α) x + α t$ jahan $x = 200, t = 255, α = 0.1$ ? ::: $205.5$ Max channel shift at $α = 0.1$ ? ::: $25.5$ levels ASR floor with $50$ equally likely classes aur koi poison nahi? ::: $1/50 = 0.02$ Kaunsa metric safety certify karta hai — clean accuracy ya ASR? ::: ASR (clean accuracy backdoor ke liye blind hai) Ek boosting attacker $n$ se scale karta hai; $n$ -client average mein effective factor kya? ::: $1.0$ (full undiluted update)

See also: Model Provenance and Supply Chain, Differential Privacy defenses ke liye jo kisi bhi single sample ka influence limit karti hain.