How Claude's Text Watermarking Actually Works

Anthropic recently outlined their intent to watermark text generated by Claude. While watermarking images or audio often involves embedding imperceptible noise directly into pixel values or frequency domains, text is discrete. You cannot simply tweak the color of a character or inject invisible metadata without it being stripped by a copy-paste into plain text.

To make a watermark persist across plain text, the signal must be baked directly into the vocabulary choices themselves.

Here is a breakdown of how statistical watermarking works during inference, how verification works, and why technical writing exposes the biggest flaws in AI detection.

How LLMs Normally Sample Tokens

Autoregressive large language models generate text sequentially, token by token. At step, given the sequence of previous tokens, the model computes raw scores (logits) across its entire vocabulary. These logits are converted via softmax into a probability distribution.

In standard generation with temperature sampling or Top-filtering, a random number generator picks the next token according to that distribution. Any plausible word has a chance of being selected based on its probability.

Source: Medium / LLM Watermark Series — KGW: A fundamental watermark for LLMs | by Kiel Dang.

How Watermarking Injects a Secret Signal

Watermarking modifies the sampling step without requiring changes to model weights. Instead of drawing from the standard distribution, the model uses a deterministic pseudo-random function tied to a private cryptographic key.

1. Vocabulary Partitioning: Before picking the next word, a keyed hash function evaluates the secret key alongside the preceding context window (typically the previous 1 to 3 tokens). This hash splits the model's vocabulary into two sets: a "green list" of preferred tokens and a "red list" of discouraged tokens.
2. Logit Biasing: The generation pipeline adds a positive bias to the logits of all tokens in the green list, or restricts sampling entirely to candidates within the green list that exceed a quality threshold.
3. Adaptive Application: In high-entropy positions, where multiple synonyms or phrasing choices are equally valid, the model easily selects a green-list token without degrading output quality. In low-entropy positions, such as strict syntax, code keywords, or medical definitions, the model falls back to the top candidate to avoid producing nonsense.

Because the choice is pseudo-randomly tied to the key and the prior context, human readers perceive normal, fluent text. Under the surface, however, the text carries an invisible statistical bias.

How Detection Works

Detection does not rely on training an opaque classifier to guess whether a paragraph "sounds like AI." Instead, it is an exact statistical test executed by a verifier who holds the private key.

1. Context Hashing: For every token in the candidate text, the verifier inspects the preceding tokens, applies the secret key, and recomputes the exact green and red lists that the generator would have produced.
2. Scoring Matches: The verifier checks whether the actual token in the text belongs to the green list. If it does, it records a match.
3. Statistical Significance: In non-watermarked human writing, tokens will fall into the green list roughly 50% of the time by pure chance. In watermarked text, the proportion of green tokens is significantly higher. By running a binomial test across the full document, the verifier computes a -score. If the score crosses a predetermined threshold, the text is flagged as AI-generated with high mathematical confidence.

The Core Problem: Low Entropy and False Positives

Statistical watermarking is far more principled than legacy heuristic detectors, but it faces fundamental practical constraints.

1. Technical Writing Has Low Entropy

In computer science papers, clinical documentation, legal contracts, or source code, vocabulary is tightly constrained. There are only so many valid ways to declare a function, cite a clinical finding, or state a mathematical theorem. When entropy is near zero, the watermarking algorithm cannot safely alter token choices without degrading accuracy. As a result, technical passages carry very weak watermark signals, leading to frequent false negatives.

2. Detection Is Centralized and Fragile

Because verification requires the secret key, third parties cannot independently verify watermarks unless Anthropic provides a public verification API. Furthermore, simple transformations, such as light paraphrasing, machine translation round-tripping, or inserting human edits, disrupt the token n-grams, rapidly eroding the statistical signal.

3. False Positives Still Hurt

When human writers happen to use phrasing that aligns with the green list, particularly in structured domains, risk of misattribution remains. In high-stakes environments like academia, medicine, and engineering, false accusations cause severe friction.

ByteByteGo explanation

ByteByteGo has a simple and straighforwad explanation that is worth taking a look.

Source: ByteByteGo newsletter.

LLMs produce text word by word. At each step, they generate probabilities for the next likely word. Instead of sampling randomly from those words, the watermarking trick changes which words are allowed to be picked.

How to watermark a response?

• Step 1: The model produces probabilities for the next word.
• Step 2: Normally a random number generator picks one of the good candidates. With watermarking, a keyed function takes a secret key plus the previous few words and decides which candidates are valid to pick from.
• Step 3: This repeats for the whole response. Places where there are multiple plausible choices carry the watermark signal.

How to detect a watermarked text?

• Step 1: For any candidate word in the text, we check whether it is a valid choice based on the secret key and the few preceding words. If the word is valid, that is counted as a match.
• Step 2: Run this across the entire text. Watermarked text matches far more often. The overall match rate can be treated as an AI-generated score.

Final Thoughts

Cryptographic watermarking is a clever engineering feat, but expecting it to reliably solve AI text attribution across the web may be unrealistic. As generative systems integrate deeper into developer workflows and research pipelines, text will increasingly be co-authored, revised, and blended.

What are your thoughts on AI watermarking and detection? Do you see it becoming an effective standard, or will the false negatives and evasion techniques create more operational headaches than solutions?