Anthropic's Claude Watermarking: How It Works and Why It's Controversial

In newly released guidance on Friday, Anthropic clarified that watermarks attach to text based on specific word choices. The company explained that large language models generate text one word at a time, choosing words probabilistically based on the preceding word. Watermarking slightly alters this random selection process. Instead of using an arbitrary random number generator, the watermark uses a key and a few preceding words to determine the next word. The chosen words remain random, but a reader with the key can check the sequence to estimate the probability that the text was generated by Claude.

John Gruber, co-creator of the Markdown markup language, criticized the approach in a blog post titled "Anthropic's 'Watermark' Text Adulteration in Claude Is a Perversion of Writing." He disagreed that these word choices are "low stakes" and argued that watermarking could corrupt the semantics of Claude's output. "The exact words we choose when writing matter," Gruber wrote. "I want any LLM I use to choose the very best, most precise words at every single decision point."

Gruber also took issue with the idea that factors other than his needs should influence text generation. "The idea that anything other than my needs should factor into the generation of text for me is patently offensive," he wrote.

Anthropic's guidance described the word choices affected by watermarking as "low stakes," citing examples like "overcast" or "gray" in a sentence about weather as effectively meaning the same thing. The company said these choices leave a pattern in generated text that is undetectable to readers but legible to those with the key.

Claude is a product owned by a tech company and was never optimized solely for the user, according to the company's critics. The backlash reflects broader concerns about over-reliance on AI tools and the importance of content provenance.