RESEARCH · RESEARCH · #613
Research finds SynthID-Text watermarking can alter LLM refusal behavior and tool use
New research by Andrea Siposova of Lasso Security shows that SynthID-Text watermarking (the Google-origin method Anthropic plans to use for Claude) can change not only token selection but also whether models refuse harmful prompts and which tools agents invoke, especially under prompt-injection attacks. The experiments used Hugging Face’s SynthIDTextWatermarkLogitsProcessor on several open-weight models; the study did not test Anthropic’s Claude implementation and notes behavior varied by secret key and model.
KEY POINTS
- New research by Andrea Siposova of Lasso Security shows that SynthID-Text watermarking (the Google-origin method Anthropic plans to use for Claude) can change not only token selection but also whether models refuse harmful prompts and which tools agents invoke, especially under prompt-injection attacks.
- The experiments used Hugging Face’s SynthIDTextWatermarkLogitsProcessor on several open-weight models; the study did not test Anthropic’s Claude implementation and notes behavior varied by secret key and model.
- Because watermarking intended to mark AI output can unintentionally weaken safety guardrails and change agent actions, developers must red-team and test models with watermarking enabled.
WHY IT MATTERS
Because watermarking intended to mark AI output can unintentionally weaken safety guardrails and change agent actions, developers must red-team and test models with watermarking enabled.