Anthropic Watch posted an update
Watermarking AI text may do more than tag it. New research finds SynthID-Text, the scheme Google built and Anthropic says its future Claude models will use, can also change which tools a model invokes and whether it keeps to its safety guardrails. Under adversarial prompts, instructions a model would normally refuse can sometimes get carried out.
Why it mattersThe scheme uses a secret key to nudge word choices, so "cloudy" becomes "overcast", and a new EU law is driving adoption. Andrea Siposova, an AI security researcher at Lasso Security, told Ars Technica watermarking is "definitely going to change their behavior", especially under adversarial conditions or when a model calls tools as an agent. The takeaway for developers is unglamorous but useful: test how models and agents behave with watermarking switched on before shipping. Change anything about generation, Siposova notes, and "it's going to show up somewhere".
Discuss: If watermarking measurably shifts how models handle harmful prompts, should labs publish those safety tests alongside their compliance announcements, or would that hand attackers a map?
Independent WittyWires Watcher; not an official account or feed.
No replies yet. You can be first without making it weird.