Watermarking and provenance: Worked examples — AI-ML deep dive

This page is a drill. We take the parent watermarking note and push its detection formula through every kind of input it can meet: strong signals, weak signals, the "no watermark at all" case, the degenerate tiny-text case, an attack that erodes the signal, and a real courtroom-style word problem.

Before line one, let us re-earn the symbols we lean on the whole way down.

Definition The symbols and the

$z$ -score, in plain words We generate $N$ tokens. A hidden rule secretly colours each token position green or red, 50/50, using a secret key. A watermarked model was nudged to prefer green words.

$N$ ::: the total number of tokens (words/word-pieces) in the text we are checking.
$n_{green}$ ::: how many of those tokens landed on their green list.
$δ$ ::: the watermark strength — the size of the nudge (added to a green token's logit) that biases the model toward green. Bigger $δ$ = more greens produced = easier to detect. It shapes the input $n_{green}$ but does not appear in the $z$ formula itself.
$p$ ::: a probability (here always a false-positive rate): the chance that pure luck — a fair coin with no watermark — would produce a $z$ -score at least this large. Small $p$ = "luck alone almost never does this" = strong evidence.
"Random baseline" ::: with no watermark, a fair coin decides green/red, so we expect $N /2$ greens.

The $z$ -score answers one question: "How many standard steps above the random baseline is our green count?" $z = \frac{n _{green} - N /2}{N /4} = \frac{2 n _{green} - N}{N}$ Big $z$ (say $> 4$ ) means "far too many greens to be luck → watermark present". $z \approx 0$ means "looks like a fair coin → no watermark".

Figure s01 — the green-count bell curve for $N = 400$ . Observe three things: the dashed navy line at the baseline $N /2 = 200$ (what a fair coin expects), the magenta double-arrow showing one "wobble" $N /4 = 10$ wide, and the orange dot for Ex 1's count of 260 greens sitting six wobbles to the right — so far out in the thin tail that no honest coin lands there.

The scenario matrix

Every case the detection formula can throw at us falls into one of these cells. The worked examples below are each tagged with the cell they cover.

Cell	Case class	What is special	Example
A	Strong watermark, large $N$	$z ≫ 4$ , clean detect	Ex 1
B	No watermark (null case)	$n_{green} \approx N /2$ , $z \approx 0$	Ex 2
C	Weak / borderline	$z$ near threshold, decision matters	Ex 3
D	Degenerate tiny $N$	too few tokens to trust	Ex 4
E	Extreme / limiting	$n_{green} = N$ (all green) or $= 0$	Ex 5
F	Attack erodes signal (paraphrase)	$z$ drops but may survive	Ex 6
G	Real-world word problem	courtroom decision + false-positive rate	Ex 7
H	Exam twist (solve for unknown)	given $z$ and $N$ , find $n_{green}$	Ex 8
I	Below baseline (anti-signal)	$n_{green} < N /2$ , negative $z$	Ex 9

Worked example Ex 1 — Cell A: strong watermark, large text

A model generates $N = 400$ tokens with watermark strength $δ = 2.0$ . Because $δ > 0$ nudges the model toward green words, we expect a green count above the fair-coin baseline. A detector counts $n_{green} = 260$ . Is it watermarked?

Forecast: guess before computing — will $z$ be small (near 0), medium (2–4), or huge ( $> 4$ )?

Compute the excess greens. $n_{green} - N /2 = 260 - 200 = 60$ . Why this step? The formula measures how far above the fair-coin expectation ( $N /2 = 200$ ) we are; everything hangs on this gap. (The $δ$ nudge is what produced these extra greens, but detection only ever sees the count $n_{green}$ , not $δ$ .)
Compute the wobble. $N /4 = 100 = 10$ . Why this step? This is one "standard step" of a 400-flip fair coin — our yardstick.
Divide. $z = 60/10 = 6.0$ . Why this step? Turning the raw gap into standard steps lets us compare against the universal threshold $z > 4$ .

Answer: $z = 6.0$ → strongly watermarked.

Verify: $6$ standard deviations above the mean of a normal curve has probability $p \approx 1 0^{- 9}$ of happening by chance. No fair coin does this. Sanity ✓.

Worked example Ex 3 — Cell C: borderline decision

$N = 100$ , $n_{green} = 62$ . Detection threshold $z_{thr} = 4$ . Watermarked or not?

Forecast: 62 out of 100 feels like a lot of greens — do you think it clears $z = 4$ ?

Excess: $62 - 50 = 12$ .
Wobble: $100/4 = 25 = 5$ .
Divide: $z = 12/5 = 2.4$ .
Compare to threshold: $2.4 < 4$ → do NOT declare watermarked. Why this step? Being above baseline is not enough; short texts wobble more, so we demand crossing a fixed bar to keep false positives low.

Answer: $z = 2.4$ → suggestive but below the bar; verdict: inconclusive/not flagged.

Verify: Notice 62/100 gave a smaller $z$ (2.4) than 260/400 (which was a smaller fraction, 65%, but gave 6.0). Fewer tokens → more forgiving of luck → higher green fraction needed. This is exactly why the $N$ sits in the denominator. Sanity ✓.

Worked example Ex 6 — Cell F: paraphrase attack erodes the signal

Original watermarked text: $N = 200$ , $n_{green} = 130$ . An attacker paraphrases, changing $40%$ of tokens. Surviving tokens keep their green status; new tokens are fair coins (50% green). Assume total length stays $200$ .

Forecast: does the watermark survive? Will $z$ stay above 4?

Count surviving tokens: $60% \times 200 = 120$ kept.
Surviving greens: the original green fraction was $130/200 = 0.65$ , so kept greens $\approx 0.65 \times 120 = 78$ . Why this step? Kept tokens retain their colour, so they carry the same green proportion.
New tokens: $40% \times 200 = 80$ new, each fair → $0.5 \times 80 = 40$ green. Why this step? Freshly written tokens carry no watermark bias, just the baseline coin.
Total greens: $78 + 40 = 118$ ; $N$ still $200$ .
$z$ : excess $= 118 - 100 = 18$ ; wobble $= 200/4 = 50 \approx 7.07$ ; $z = 18/7.07 \approx 2.546$ .

Answer: $z \approx 2.55$ — the watermark weakened (was $z = \frac{130 - 100}{7.07} \approx 4.24$ ) and now falls below the $z > 4$ bar.

Verify: Pre-attack $z = 30/7.07 \approx 4.24$ (detectable). Post-attack $2.55 < 4$ (evades a strict detector). This confirms the parent's warning: paraphrasing dilutes but does not fully erase — mitigation is longer hash context or larger $δ$ . Sanity ✓.

Worked example Ex 7 — Cell G: real-world courtroom word problem

A publisher is accused of passing AI text as human. The disputed article has $N = 900$ tokens; the platform's detector finds $n_{green} = 510$ . Policy: flag only if $z > 4$ . Should the court treat this as strong evidence?

Forecast: 510 of 900 is only 56.7% green — is that enough over 900 tokens?

Excess: $510 - 450 = 60$ . Why? Baseline for 900 tokens is $N /2 = 450$ .
Wobble: $900/4 = 225 = 15$ .
$z$ : $60/15 = 4.0$ .
Turn $z$ into a false-positive rate $p$ . The exact one-sided normal-tail probability of $z \geq 4$ is $p \approx 3.2 \times 1 0^{- 5}$ — about $3$ in $100, 000$ . Why this step? A court cares not about "flagged/not flagged" but about the chance of a wrong accusation — that chance is the $p$ -value. Note: the parent note's phrase " $z = 4$ gives $p < 0.003$ " is a deliberately loose, safe upper bound (any bound $< 0.003$ is honoured, and the true tail is far smaller). Both statements agree: $3.2 \times 1 0^{- 5} < 0.003$ . There is no contradiction — one is a conservative bar, the other the exact number.

Answer: $z = 4.0$ , exact $p \approx 3.2 \times 1 0^{- 5}$ ; evidence is strong and clears the stated policy, but sits right at the flag threshold, so treat as corroborating alongside signed provenance rather than as sole proof. (See AI regulation and Cryptographic signatures for why watermarks pair with signatures, not stand alone.)

Verify: $p (z \geq 4) \approx 3.17 \times 1 0^{- 5}$ from the standard normal tail, and $3.17 \times 1 0^{- 5} < 0.003$ , so both the exact value and the parent's bound point the same way. Sanity ✓.

Worked example Ex 9 — Cell I: below baseline (negative

$z$ ) A text has $N = 100$ , $n_{green} = 38$ (fewer greens than a coin would give). What does $z$ say?

Forecast: can $z$ be negative, and what would that even mean?

Excess: $38 - 50 = - 12$ (below baseline).
Wobble: $100/4 = 5$ .
Divide: $z = - 12/5 = - 2.4$ . Why this step? The sign is information: a large negative $z$ means the text avoids green tokens — either bad luck, or a deliberate anti-watermark attack that flips toward red.

Answer: $z = - 2.4$ → no watermark of this key; and a strongly negative $z$ across many texts hints someone is scrubbing the signal (relates to Adversarial examples).

Verify: $- 2.4$ is $2.4$ standard steps below the mean — as ordinary-to-mildly-unusual on the low side as Ex 3 was on the high side. Detectors only flag the positive tail ( $z > z_{thr}$ ), so this text is correctly not flagged as watermarked. Sanity ✓.

One picture tying the cells together

Figure s02 — every worked example placed on the standardized $z$ -axis. The magenta vertical line at $z = 4$ is the decision boundary; the shaded magenta sliver to its right is the "flag as watermarked" zone. Watch how the honest cases (Ex 2 at $0.5$ , Ex 9 at $- 2.4$ ) sit under the fat middle of the bell, the borderline cases (Ex 4 at $2.0$ , Ex 3 at $2.4$ ) hover left of the line and stay unflagged, while Ex 7 ( $4.0$ ) lands exactly on the boundary and Ex 1 ( $6.0$ ) sits deep in the flag zone. Ex 5's ceiling ( $z = 20$ ) is far off the right edge — the strongest a fair coin could ever be beaten.

Look at the sunset bell curve: the threshold line at $z = 4$ (magenta) is the decision boundary. Ex 2 ( $z = 0.5$ ) and Ex 9 ( $z = - 2.4$ ) sit under the fat middle — ordinary. Ex 3 ( $2.4$ ) and Ex 4 ( $2.0$ ) sit in the "suspicious but not proven" band. Ex 1 ( $6$ ), Ex 7 ( $4.0$ ), and the ceiling Ex 5 ( $20$ ) live in the far-right sliver where a fair coin essentially never lands.

Recall Quick self-test

For $N = 400$ , $n_{green} = 260$ , what is $z$ ? ::: $z = (260 - 200) /10 = 6.0$ . Why can't a 4-token all-green text be flagged? ::: Its $z$ is only $2.0$ ( $p = 1/16$ ), far short of the $z > 4$ bar; tiny samples wobble too much. What is the maximum $z$ for $N = 400$ ? ::: $400 = 20$ , reached when every token is green. A text gives $z = - 2.4$ . Watermarked? ::: No — negative $z$ means fewer greens than a coin; detectors flag only the positive tail. Does $δ$ appear in the $z$ formula? ::: No — $δ$ shapes how many greens the model produces, but detection reads only $n_{green}$ and $N$ .

Related deep material: AI-generated content detection, Model fingerprinting, Content moderation, and the parent Watermarking and provenance (Hinglish).