Watermarking and provenance: Foundations — Hinglish deep dive

6.4.12 · D1 · AI-ML › AI Safety & Alignment › Watermarking and provenance

Is page par assume kiya gaya hai ki aapko kuch bhi pata nahi. Parent note padhne se pehle, aapko har woh symbol samajhna hoga jo wahan use hota hai. Hum har ek ko ek picture se build karenge, explain karenge ki topic ko uski zaroorat kyun hai, aur tabhi usse appear hone denge.

1. Ek token — text ka atom

Picture: ek train imagine karo. Har dabba ek token hai, order mein coupled. Model ek dabba ek baar mein banata hai, aur agla choose karne se pehle hamesha jo dabba usne abhi jooda usse dekhta hai.

Topic ko iski zaroorat kyun hai. Text watermarking generation time par hoti hai — theek usi waqt jab model agla dabba $w_{t}$ pick karta hai. Agar aap text ko ordered chain of choices ke roop mein nahi sochte, toh baad ke steps mein se koi bhi samajh nahi aayega. Dekhein Information theory ki kyun "ek choice" hidden information ki unit hai.

2. Vocabulary — sabhi possible tokens ka dabba

Picture: Scrabble tiles ka ek bada bag. Har step par model usi same bag mein haath daalta hai aur ek tile nikalti hai. Bag kabhi nahi badlta; bas kaun si tile nikli woh badlti hai.

Topic ko iski zaroorat kyun hai. Watermark is bag ko do hisson mein split karke kaam karta hai (ek "green" half aur ek "red" half) aur quietly ek half ko prefer karta hai. Kisi cheez ko split karne ke liye, pehle yeh jaanna zaroori hai ki poori cheez kya hai.

3. Logits — model ke raw scores

Picture: 50,000 runners ki ek race jismein sab line up hain. Har runner ke paas ek number pinned hai — wahi uska logit hai. Jitna bada number, utna aage se start karta hai.

4. Softmax — scores ko probabilities mein convert karna

Hamare paas scores hain, lekin actually ek token pick karne ke liye hume probabilities chahiye (0 aur 1 ke beech ke numbers jo milke 1 hote hain). Woh tool jo scores ko probabilities mein convert karta hai use softmax kehte hain.

Is line ke har symbol ko samajhte hain.

$e^{x}$ kyun, raw score kyun nahi? Do reasons hain, aur dono matter karte hain:

Scores negative ho sakte hain; probabilities nahi ho sakti. $e^{x}$ hamesha positive hai, isliye har token ko ek sensible non-negative chance milti hai.
Yeh $δ$ add karne ke effect ko multiplicative banata hai. Dekho:

Picture: softmax ek "vote-share calculator" hai. Har runner ke number ko $e^{(\cdot)}$ do, fir har runner ki probability total pie mein uska apna slice hai. Green runners ko $δ$ se bump karna unke slices ko bada karta hai.

5. δ — watermark strength (woh nudge)

Topic ko iski zaroorat kyun hai. $δ$ ek single dial hai jo detectability aur quality ke beech trade karta hai. Parent mein har debate (poetry degradation, paraphrase attacks) actually ek debate hai ki $δ$ kitna bada banayein. Related idea: model ke output ko nudge karna Adversarial examples ka close cousin hai, jahan ek choti push behaviour badal deti hai.

6. Hashing aur secret key — reproducible randomness

Picture: ek coffee grinder. Wahi beans daalo, hamesha wahi grounds niklengi — lekin grounds se beans kabhi rebuild nahi kar sakte.

Topic ko iski zaroorat kyun hai. Har position par green/red split:

har jagah alag honi chahiye (taaki pattern obvious na ho) — achieve hota hai kyunki input mein $w_{t - 1}$ included hai;
reproducible honi chahiye (taaki verifier baad mein ise recompute kar sake) — achieve hota hai kyunki hash deterministic hai;
secret honi chahiye (taaki sirf owner check kar sake) — achieve hota hai $secret_key$ mix karke.

7. Green list aur red list — bag ko split karna

Picture: har step par, tiles ka bag repaint hota hai — aadhi tiles green ho jaati hain, aadhi red rehti hain — aur painting pattern decide hota hai previous carriage $w_{t - 1}$ se.

Topic ko iski zaroorat kyun hai. Watermark green ke preference ke alawa kuch nahi hai. Natural (un-watermarked) text mein, pure chance se lagbhag aadhe tokens green lagte hain. Watermarked text mein, adhe se noticeably zyada green hote hain. Yahi gap detector dhundta hai. "Model ne banaya" ko random baseline se distinguish karna exactly AI-generated content detection ka goal hai.

8. Counting aur z-score — prove karna ki yeh chance nahi hai

Ab hume prove karna hai ki ek suspicious text sach mein watermarked hai, na ki bas lucky. Hum green tokens count karte hain aur poochte hain: kya yeh count surprising hai?

Picture: 200 coins flip karo, aap expect karte ho ~100 heads lekin usually 100 ± 7 ke aas-paas rahoge. 150 heads ka result cheekh kar bolega "yeh coins rigged hain." Watermarked text woh rigged coin hai.

Yeh exact formula kyun? $μ$ subtract karna count ko zero par centre karta hai (taaki "koi surprise nahi" = 0 ho). $σ$ se divide karna surprise ko ek universal unit mein convert karta hai ("wobbles ki sankhya"), isliye wahi threshold 50-token text aur 5000-token dono ke liye kaam karta hai. Yahi standardising move hai jo formula ke dono forms ko equal banati hai (bas simplify karo $N /4 = \frac{N}{2}$ ).

Prerequisite map

Neeche di gayi figure dikhati hai ki har foundation agla kaise feed karta hai, sab parent topic ki taraf flow karte hue.

Equipment checklist

Parent padhne se pehle self-test karo. Daaya side cover karo aur jawab do.

Token

w_{t}

kya hai?

Text ka sabse chota chunk (word ya word-piece) jo model pick karta hai, position

t

par.

w_{< t}

ka kya matlab hai?

Position

t

se pehle likhe gaye saare tokens — abhi tak ka poora context.

Vocabulary aur uski size

V

kya hai?

~50,000 candidate tokens ka fixed box jismein se model choose karta hai;

V

batata hai kitne hain.

Logit kya hai?

Token ke liye model ka raw score — koi bhi real number, zyada = zyada preferred.

Hum probabilities ki jagah logits ke saath kyun kaam karte hain?

Kyunki aap score mein nudge

δ

cheaply add kar sakte ho, lekin probability mein nahi.

P (w_{t} ∣ w_{< t})

ka kya matlab hai?

Agla token ki probability given abhi tak likha gaya sab kuch — ek conditional probability.

Softmax kya karta hai?

Scores ko probabilities mein convert karta hai

P (w_{t} ∣ w_{< t}) = e^{logit (w_{t})} / \sum_{w^{'}} e^{logit (w^{'})}

ke zariye, taaki woh positive hon aur sum 1 ho.

e

score ke saath kya karta hai?

Use positive banata hai aur

δ

add karne ko

e^{δ}

se multiply karne mein convert karta hai.

\sum_{w^{'}}

ka kya matlab hai?

"Ise har token par add karo" — woh normaliser jo probabilities ka total 1 force karta hai.

δ

kya hai?

Watermark strength: woh amount jo green-list logits mein add hoti hai, quality vs detectability trade karte hue.

Secret key kya hai?

Ek fixed private seed jo har hash mein mix hoti hai, taaki sirf uska holder lists recompute aur verify kar sake.

Vocabulary exactly green aur red mein kaise split hoti hai?

h_{t}

se ek shuffler seed karo, vocabulary shuffle karo, pehle

⌊ γ V ⌋

tokens green lo.

Odd-sized vocabulary ko kaise split karein?

Green fraction

γ

use karo aur

⌊ γ V ⌋

tokens green lo; one-token imbalance negligible hai.

Pehle token par kya hota hai jahan

w_{t - 1}

nahi hota?

Ek fixed starting seed use karo (key alone ya ek dummy

w_{0}

), ya token 1 unwatermarked chhod do aur

t = 2

se detection start karo.

Watermark ke bina, kitne fraction tokens green hote hain?

Lagbhag aadhe — ek fair-coin 50%.

N

fair coin flips ke liye

μ

aur

σ

kya hain?

Mean

μ = N /2

, standard deviation

σ = N /4

Z-score kya measure karta hai?

Kitne standard deviations green count random baseline se upar baitha hai.

N = 200

n_{green} = 125

ke liye

z

compute karo.

z = (2 \cdot 125 - 200) / 200 \approx 3.54