6.4.12 · D5AI Safety & Alignment
Question bank — Watermarking and provenance
A tiny reminder of the vocabulary, so nothing below sneaks in undefined:
Recall The words this page assumes
A watermark is a hidden statistical pattern woven into the content as it is generated. Provenance is the recorded history (who/when/what tools). A logit is the raw score a model gives a token before it is turned into a probability; adding a bias to some logits tilts the model toward those tokens. The green list is the set of tokens secretly promoted at a given position; the z-score counts how many standard deviations the green-token count sits above the "random coin-flip" baseline of .
If any of those feel shaky, the derivations live in the parent note — come back after.
True or false — justify
A watermark is stored in the file's metadata.
False. Metadata (like C2PA tags) sits beside the content and can be stripped; a watermark is woven into the token choices or pixels themselves, so it survives copying and re-saving. This distinction is the whole reason both mechanisms exist. See Cryptographic signatures for the metadata side.
Two people asking the same model the same question get watermarks that look identical.
False. The green list at each position is seeded by the previous tokens plus a secret key, so as soon as the two outputs diverge by one word, every downstream green list diverges too — the signal is content-dependent, not a fixed stamp.
A longer watermarked text is easier to detect than a short one.
True. The z-score grows like for a fixed green-fraction, so more tokens means more standard deviations of separation from the random baseline. Ten watermarked tokens are basically undetectable; a thousand are unmistakable.
If a detector reports a high z-score, the text is definitely AI-generated.
False. High z means the pattern is statistically very unlikely by chance, not impossible. A false positive rate of still fires once in ~300 innocent documents, so "very strong evidence" is not "proof" — this is why AI regulation treats detectors as one input, not a verdict.
Raising always makes a watermark strictly better.
False. Higher forces the model to pick green tokens more often, which sharpens detection but distorts word choice — perplexity rises and humans notice awkward phrasing. It's a trade-off, not a free win.
A watermark with still leaves a faint detectable trace.
False. With the green tokens get no boost, so each position is a genuine 50/50 coin flip — the expected green count is exactly and z hovers around zero. No bias means no signal at all.
C2PA and green-list watermarking solve the same problem.
False and worth separating. C2PA asserts provenance via a signature that breaks if content changes; green-list watermarking hides an intrinsic signal that survives edits but cannot name the author. One is a tamper-evident label, the other is invisible ink.
Spot the error
"We used a secret key, so an attacker who reads our text still can't tell it's watermarked."
The error: reading the text is not the threat — the key protects verification, but the statistical bias itself can be estimated by an attacker who queries the model enough. Secrecy of the key stops forgery of the detector, it does not make the signal information-theoretically invisible (see Information theory).
"Paraphrasing changes 40% of the words, so exactly 40% of the watermark signal is lost."
The error: the loss is worse than proportional. Because each green list depends on the previous token, changing one word poisons the green list at the next position too — you lose the changed tokens and corrupt the ones after them.
"The detector needs the original model's weights to check a watermark."
The error: it needs the secret key and the hashing rule, not the weights. Detection just re-derives each position's green list from the previous tokens and counts hits — a cheap, model-free scan.
"Image watermarks and text watermarks both add small Gaussian noise to the output."
The error: text is discrete — you cannot add a fraction of a word. Text watermarking biases the choice among tokens; only continuous media (images, audio) can absorb literal additive noise. Confusing the two is the classic beginner slip.
"Our watermark is robust because the detector is trained against JPEG compression."
The error: robustness to JPEG doesn't imply robustness to a paraphrase or a targeted removal attack. Defending against known transformations is not defending against an adaptive adversary — see Adversarial examples.
"If a photo has no C2PA signature, it must be a fake."
The error: absence of a signature only means the tool didn't sign it — most legitimate cameras and old images have none. Missing provenance is a reason to reduce trust, not to conclude forgery.
Why questions
Why seed the green list with the previous token instead of just the token position?
Because provenance must survive edits like insertion or deletion. Position-based seeding would desynchronise the moment one token is added; content-based seeding keeps each position's list tied to its actual local context. It also makes the pattern context-dependent and harder to strip.
Why is the detection statistic a z-score rather than just "count the green tokens"?
Because a raw count of, say, 125 is meaningless without knowing and the expected spread. The z-score normalises the surplus by the standard deviation , turning it into a scale-free "how surprising is this?" number comparable across text lengths.
Why does the parent note say watermarks are "like seatbelts, not brakes"?
Because they help you detect and respond after AI content exists, but do nothing to prevent its creation — an open-source model or a determined attacker sidesteps them entirely. They are one layer of a defence stack that also includes Content moderation and policy.
Why can a cryptographic signature prove authorship while a watermark generally cannot?
A signature is made with a private key that only the author holds, so a valid signature ties content to that key. A green-list watermark only proves "a watermarked model made this," not which person ran it — it lacks the identity binding of Cryptographic signatures.
Why does watermarking hurt poetry more than casual chat?
Poetry lives on precise, sometimes rare word choices; forcing green-list tokens shrinks the usable vocabulary right where it matters most. Casual chat has many interchangeable phrasings, so the bias barely dents quality. Same , very different damage.
Why is watermarking different from Model fingerprinting?
A watermark is deliberately planted in the output to be detected later; a fingerprint is an unintentional statistical quirk of a model's behaviour that identifies the model even when nobody tried to mark anything. One is a signature you write; the other is a scent you can't help leaving.
Edge cases
What is the z-score of a purely human-written text that happens to use many "green" words by luck?
It fluctuates around 0, occasionally spiking up. That spike is exactly the false-positive risk — the detector cannot distinguish a lucky human from a weak watermark, which is why the threshold is set high (z ≈ 4).
What happens to detection if the text is only 5 tokens long?
Detection collapses. With the standard deviation is large relative to the possible surplus, so even an all-green text barely clears z ≈ 2 — statistically inconclusive. Watermarks need length to speak.
A user regenerates the same prompt 50 times and averages nothing — just picks the least awkward output. Does the watermark survive?
Yes, each individual output is still watermarked; cherry-picking among watermarked drafts doesn't erase the bias, it just selects a fluent one. Averaging across tokens would break it, but you can't average discrete text.
What if an attacker inserts one random green-neutral word every few tokens?
This is a cheap desync attack: each insertion shifts the context that seeds the next green list, corrupting the signal for the following tokens. Robust schemes counter it by hashing over a longer, more forgiving context window.
Content is watermarked and C2PA-signed, then someone crops the image. What survives?
The C2PA signature breaks (its hash no longer matches the altered pixels), while a robustly-trained pixel watermark can survive the crop. This is precisely why the two mechanisms are stacked rather than treated as substitutes.
Can a watermark detector work on content the model did not generate but that mimics its style?
No — mimicry copies surface style, not the secret green-list bias, so the mimic's z-score stays near zero. This is a feature: watermark detection tests for the planted pattern, not for "does this look AI-ish," which is the job of AI-generated content detection.
If detection is public (anyone can run it), why isn't removal trivial?
Public detection needs the secret key, and even with it, removal means changing enough tokens to push z below threshold — which degrades the text and requires knowing the exact green lists. Detectability and removability are separate difficulties.