Watermarking and provenance

What is Watermarking?

How Text Watermarking Works (Derivation)

The Challenge: Text is discrete (words/tokens), not continuous like images. We can't just add noise. We need a method that:

• Works at generation time (when the model picks tokens)
• Doesn't noticeably change the text quality
• Is statistically detectable

The Green-Red List Method (Kirchenbauer et al., 2023)

Step 1: Partition the vocabulary

For each token position $t$, use the previous token(s) as a seed to hash the vocabulary into two sets:

• Green list $\mathcal{G}_t$ (promoted tokens)
• Red list $\mathcal{R}_t$ (demoted tokens)

$h_t = \text{HASH}(w_{t-1}, \text{secret\_key})$

Use $h_t$ to partition: if $\text{HASH}(w_i, h_t) \bmod 2 = 0$, then $w_i \in \mathcal{G}_t$.

Why this step? We need randomness that depends on context (so it's different for each position) but is reproducible (so we can verify later). The secret key ensures only authorized parties can verify.

Step 2: Bias the sampling

When the model generates token $t$, modify the logits before sampling:

\text{logit}(w) + \delta & \text{if } w \in \mathcal{G}_t \\ \text{logit}(w) & \text{if } w \in \mathcal{R}_t \end{cases}$$ where $\delta > 0$ is the ==watermark strength== (typically $\delta = 2.0$). **Why this step?** Adding $\delta$ to green tokens increases their probability via the softmax: $$P(w) = \frac{e^{\text{logit}'(w)}}{\sum_{w'} e^{\text{logit}'(w')}}$$ If $\text{logit}'(w) = \text{logit}(w) + \delta$, then $P'(w) = P(w) \cdot e^\delta / Z$ where $Z$ is a normalization factor. For $\delta=2$, this bosts probability by $\sim e^2 \approx 7.4times$. **Step 3: Detection** Given a text of $N$ tokens, count how many are from their respective green lists: $$z = \frac{|\{t : w_t \in \mathcal{G}_t\}| - N/2}{\sqrt{N/4}}$$ **Why this formula?** Under the null hypothesis (no watermark), each token has50% chance of being green (random partition). This is a binomial distribution: - Mean: $\mu = N/2$ - Variance: $\sigma^2 = N \cdot (1/2) \cdot (1/2) = N/4$ The $z$-score measures standard deviations above the expected random baseline. For watermarked text, $z \gg 0$ (typically $z > 4$ is very strong evidence). ![[6.4.12-Watermarking-and-provenance.png]] >[!formula] Watermark Detection Threshold >$$z = \frac{n_{\text{green}} - N/2}{\sqrt{N/4}} = \frac{2n_{\text{green}} - N}{\sqrt{N}}$$ >Detect watermark if $z > z_{\text{threshold}}$ (e.g., $z_{\text{threshold}} = 4$ gives $p< 0.003$ false positive rate). > >**Derived from**: Central Limit Theorem applied to binomial distribution. As $N \to \infty$, the distribution of green-list tokens approaches $\mathcal{N}(N/2, N/4)$. ## Image Watermarking **Two main approaches**: ### 1. Latent Diffusion Watermarking Inject a signal into the latent space during generation: $$z_t = z_t + \alpha \cdot w$$ where $w$ is a learned watermark pattern, $\alpha$ is strength. **Why it works**: The decoder is trained to preserve this pattern through the image generation process. Extraction uses a trained neural network that recognizes the pattern in the output image. ### 2. Stable Signature (Meta's approach) Add imperceptible perturbations to pixel space that survive image transformations: $$I' = I + \epsilon \cdot \text{sign}(\nabla_I L_{\text{detect}})$$ where $L_{\text{detect}}$ is a loss that ensures a detector network can extract a specific message. **Robustness**: Trained adversarially against JPEG compression, cropping, blurring, etc. >[!example] Text Watermarking Example >**Scenario**: GPT generates a 200-token essay with $\delta = 2.0$ watermark strength. >**Generation**: >- Token 1: Previous context → hash → 50% of vocab is green. "The" is green (boosted), selected. >- Token 2: "The" → new hash → different green list. "cat" is green (boosted), selected. >- ... (repeat) > >**Detection**: >- Count green tokens: 125 out of 200 >- $z = (125 - 100) / \sqrt{50} = 25 / 7.07 \approx 3.54$ >- **Interpretation**: 3.54 standard deviations above random → $p = 0.0002$ → strong evidence of watermark. > >**Why this detection works**: Random text would have $\sim 100 \pm 7$ green tokens. Seeing 125 is statistically extremely unlikely without watermarking. >[!example] Attack Scenario: Paraphrasing >**Attack**: User takes watermarked text, feeds it to another AI to paraphrase. > >**Question**: Does the watermark survive? >**Analysis**: >- If paraphraser changes40% of tokens, we have120 original tokens left >- Of those, ~75 were green (if original had 125/200) >- Plus 40new tokens with random50% green → +20 >- Total: 95 green out of 160 tokens >- $z = (95 - 80) / \sqrt{40} = 15/6.32 \approx 2.37$ >**Result**: Still detectable, but weaker. **Why**: Green-list positions depend on previous tokens—when we change tokens, we change which positions expect green tokens, breaking some of the signal. > >**Mitigation**: Use longer context windows for hashing (more robust) or higher $\delta$ (stronger signal). >[!mistake] Mistake: "Watermarks prevent misuse" >**The intuitive (wrong) idea**: "If we watermark all AI content, people can't misuse AI for fake news." > >**Why it feels right**: Detection = prevention, right? > >**The reality**: Watermarks only help *detect* AI content after it's created. They don't prevent: >- Someone from removing the watermark (adversarial attack) >- Open-source models without watermarking >- Using AI for private harmful content (no one checks) > >**The fix**: Watermarks are *one layer* of defense. We also need: >- Platform policies (require watermarking for publishing) >- Education (media literacy) >- Legal frameworks (liability for unmarked AI content in certain contexts) > >Think of watermarks like seatbelts: they help if something goes wrong, but don't prevent the accident. >[!mistake] Mistake: "Watermarks don't degrade quality" >**The intuitive (wrong) idea**: "Adding $\delta = 2$ to logits is tiny, humans won't notice." > >**Why it feels right**: In a vocabulary of 50k tokens, biasing 25k by $\delta=2$ seems negligible. > >**The reality**: Quality *does* degrade, especially for: >- Poetry/creative writing (restricted word choice) >- Technical content (forcing less precise terms) >- Languages with smaller vocabularies > >**Measurement**: Perplexity increases by 5-15%. Human evaluators notice "slightly more awkward" phrasing in 20-30% of cases. >**The tradeoff**: Lower $\delta$ → better quality but harder detection. Higher $\delta$ → easy detection but worse quality. Optimal $\delta$ depends on use case (chatbot vs. creative writing). ## Provenance Systems ### C2PA (Coalition for Content Provenance and Authenticity) A standard for attaching cryptographically-signed metadata: $$\text{Manifest} = \{ \text{content}, \text{author}, \text{timestamp}, \text{edits}, \text{signature} \}$$ **How signing works**: 1. Hash the content: $h = \text{SHA256}(\text{content})$ 2. Sign with private key: $s = \text{Sign}_{\text{private}}(h)$ 3. Attach $(h, s, \text{public\_key})$ as metadata 4. Verification: $\text{Verify}_{\text{public}}(h, s) = \text{TRUE/FALSE}$ **Why this works**: Cryptographic signatures ensure that if the content OR metadata changes, verification fails. This proves "this content came from this source at this time." **Limitation**: Requires adoption by platforms and cameras/tools. Metadata can be stripped (though that itself is detectable—"why is there no signature?"). >[!example] Provenance Chain Example >**Scenario**: A news photo goes through multiple edits. > >**Step 1**: Camera captures image → signs with C2PA manifest >- Manifest: {content_hash: 0x3f2a.., device: "Canon EOS R5", timestamp: 2026-07-01T00:30:00Z, signature: 0x8b4c...} > >**Step 2**: Editor crops and adjusts color → adds to chain >- New manifest: {previous_hash: 0x3f2a..., edits: ["crop(100,100,500)", "brightness(+10)"], editor: "Adobe Photoshop", timestamp: 2026-07-01T02:15:00Z, signature: 0x2d9f...} > >**Step 3**: AI upscales resolution → adds to chain >- New manifest: {previous_hash: 0x2d9f..., edits: ["ai_upscale(model=Gigapixel)"], timestamp: 2026-07-01T02:45:00Z, signature: 0x7e1a...} > >**Verification**: Each step's signature confirms authenticity. Viewer sees complete history: "Original photo → cropped by human → AI-upscaled." Trust is maintained because nothing is hidden. ## Types of Watermarks | Type | Method | Robustness | Quality Impact | |------|------------|-------------| | ==Statistical== (text) | Biased token sampling | Medium (vulnerable to paraphrase) | Low-Medium | | ==Latent== (images) | Embedded in generation space | High (survives transforms) | Very Low | | ==Perceptual== (images) | Pixel-space perturbations | Very High (trained adversarially) | Low | | ==Cryptographic== Signed hashes/metadata | Perfect (unless stripped) | None (separate) | ## Why Watermarking is Hard **The fundamental tension**: $$\text{Detectability} \propto \text{Signal Strength} \propto \text{Quality Degradation}$$ We want high detectability with zero quality loss—but information theory says we can't have both perfectly. **Attack surface**: 1. **Removal attacks**: Adversarial networks trained to remove watermarks 2. **Substitution attacks**: Replace watermarked content with non-watermarked version 3. **Collusion attacks**: Mix outputs from multiple models to dilute watermarks 4. **Evasion**: Simply use models without watermarks (open-source) **The arms race**: Watermark designers make them robust → attackers find weaknesses → designers patch → attackers adapt. There's no "perfect" watermark, only "good enough for now." >[!recall]- Explain to a 12-year-old >Imagine you're making cookies with a special cookie cutter that leaves a tiny mark on the bottom—like a star pattern. When you look at the cookie normally, you can't see it. But if you use a magnifying glass, there it is! > >That's what watermarking AI content is like. When an AI writes a story or makes a picture, we can put a secret pattern in it. Humans can't tell it's there, but special computer programs can look and say "Yep, an AI made this!" > >Why do we need this? Well, imagine if someone wrote fake news stories using AI and pretended they were real. Or made fake videos of people saying things they never said. Watermarks help us know "this came from an AI" so we're not fooled. > >But here's the tricky part: bad guys try to remove the watermarks, like washing the star pattern off the cookie bottom. So the people who make watermarks have to be really clever and hide the pattern so well that it can't be washed off, even if someone tries really hard. >[!mnemonic] WATERMARK >**W**hy: distinguish AI from human >**A**lgorithm: bias token/pixel selection >**T**rade-off: detectability vs. quality >**E**vasion: attacks try to remove it >**R**obustness: survives modifications >**M**etadata: separate approach (C2PA) >**A**rms race: continuous improvement >**R**egulation: may become legally required >**K**ey: secret parameter for verification ## Real-World Applications 1. **Education**: Detect AI-written essays 2. **News media**: Verify image authenticity 3. **Legal**: Prove document provenance 4. **Social media**: Label AI-generated content 5. **Copyright**: Track AI-created art usage ## Open Problems - **Adversarial robustness**: Can we make watermarks unremovable? - **Multi-modal**: Watermark text+image+audio together? - **Privacy**: Watermarks might leak info about the model or user - **Standardization**: Need industry-wide adoption (currently fragmented) --- ## Connections - [[Model fingerprinting]] - related technique to identify which model created content - [[AI-generated content detection]] - broader problem watermarking helps solve - [[Adversarial examples]] - techniques used to attack watermarks - [[Cryptographic signatures]] - complement to watermarks for provenance - [[Content moderation]] - watermarks help platforms enforce policies - [[AI regulation]] - watermarking may become legally mandated - [[Information theory]] - fundamental limits on watermark capacity --- ## #flashcards/ai-ml What is a watermark in AI content? :: A detectable signal embedded in AI-generated content that survives modifications, remains imperceptible to humans, and proves provenance. How does the green-red list watermarking method work? ::: Partition vocabulary into green (promoted) and red (demoted) lists based on previous tokens + secret key, boost green logits by δ during generation, detect by counting green tokens. What is the z-score formula for watermark detection? ::: z = (n_green - N/2) / sqrt(N/4), where n_green is count of green tokens and N is total tokens. Measures standard deviations above random baseline. Why can't watermarks be perfect (fundamental limit)? ::: Information theory: detectability is proportional to signal strength, which is proportional to quality degradation. Can't maximize all three simultaneously. What is provenance in AI content? ::: The complete history of content: who created it, when, with what tools, and what modifications were made—the "chain of custody." What is C2PA? ::: Coalition for Content Provenance and Authenticity—a standard for attaching cryptographically-signed metadata to prove content origin and edit history. Name three types of attacks on watermarks :: (1) Removal attacks (adversarial networks), (2) Substitution attacks (replace with non-watermarked), (3) Collusion attacks (mix multiple model outputs). Why is paraphrasing a challenge for text watermarks? ::: Changing tokens changes which positions expect green tokens (hash depends on previous tokens), breaking part of the watermark signal and reducing detection z-score. What's the difference between watermarks and metadata? ::: Metadata can be stripped easily; watermarks are woven into the content itself and survive modifications like cropping or compression. What does a z-score of 4 mean in watermark detection? ::: 4 standard deviations above the expected random baseline, corresponding to p < 0.00003 false positive rate—very strong evidence of watermarking. ## 🖼️ Concept Map ```mermaid flowchart TD P[Provenance chain of custody] -->|enables trust in| C[AI content problem] W[Watermarking] -->|proves| P W -->|embedded in| CN[Content itself] CN -->|survives| M[Modifications] W -->|imperceptible unlike| MD[Metadata can be stripped] W -->|implemented via| GR[Green-Red list method] GR -->|Step 1 partition| V[Vocabulary hashed by prev token] V -->|uses| K[Secret key seed] GR -->|Step 2 bias logits| D[Add delta to green tokens] D -->|boosts probability via| S[Softmax sampling] GR -->|Step 3| DET[Detection z-score] DET -->|counts| G[Green list tokens over N] ``` ## 🔊 Hinglish (regional understanding) > [!intuition]- Hinglish mein samjho > AI watermarking ka matlab hai ki jab koi AI content generate karta hai—chahe text ho, image ho, ya video—toh usme ek invisible signature daal dete hain. Yeh signature normal insaan ko dikhta nahi, lekin special detector software isko pakad leta hai. Sochiye aise jaise currency notes mein security thread hoti hai—aam admi ko pata nahi chalti, lekin bank ka machine turant detect kar leti hai. Yeh zaroori kyun hai? Kyunki ajkal AI itna advanced ho gaya hai ki fake news, deepfakes, aur AI-written content ko real se distinguish karna mushkil ho gaya hai. Agar students AI se essays likhwa lein ya koi fake video banaye, toh watermarking se pata chal jayega ki "yeh AI ne banaya hai." > > Text watermarking mein technique yeh hoti hai: jab AI koi sentence generate kar raha hota hai, toh har token (word) ke liye vocabulary ko do groups mein divide karte hain—green list aur red list. Green list ke words ko thoda boost dete hain (unka probability badhate hain). Phir jab detection time aata hai, toh count karte hain kitne green words use hue. Agar bohot zyada hain (matlab expected se 3-4 standard deviations upar), toh confirm ho jata hai ki "yeh watermarked text hai." Lekin challenge yeh hai ki quality maintain karni padti hai—agar watermark bohot strong banayenge, toh text awkward lagega. Isliye ek balance banana padta hai detectability aur quality ke bech. Image watermarking mein bhi same concept hai, bas technique thodi alag hoti hai—latent space mein ya pixel level pe patterns inject karte hain jo transformations (cropping, compression) survive kar sakein. > > Provenance tracking ka concept alag hai—yeh metadata ka use karta hai jisme cryptographic signatures hote hain. Jaise koi document ka complete history track karna: kis camera se photo li, kisne edit kiya, kaunsa AI model usehua, sab kuch signed record hota hai. C2PA standard yeh kaam karta hai. Lekin dikat yeh hai ki metadata strip kiya ja sakta hai, isliye watermarking usse better hai kyunki woh content mein hi embed hota hai. Abhi industry mein dono approaches chalte hain—sometimes together. Future mein shayad regulation aaye ki sab AI companies ko compulsory watermarking lagana padega, especially news aur social media platforms pe. ![[audio/6.4.12-Watermarking-and-provenance.mp3]]