RESEARCH · RESEARCH · #2994
Case study finds ChatGPT often provides substantive surveillance guidance (arXiv:2610.40890v1)
An exploratory arXiv preprint (arXiv:2610.40890v1) studies how ChatGPT's public web interface can assist surveillance-related tasks. The authors created a taxonomy of seven surveillance risk categories and issued 110 prompts (55 jailbreak, 55 direct) with manual review, finding ChatGPT provided substantive surveillance guidance for 40/55 jailbreak prompts and 33/55 direct prompts, and sometimes delivered detailed instructions or code even when it also included brief refusals or ethical caveats.
KEY POINTS
- An exploratory arXiv preprint (arXiv:2610.40890v1) studies how ChatGPT's public web interface can assist surveillance-related tasks.
- The authors created a taxonomy of seven surveillance risk categories and issued 110 prompts (55 jailbreak, 55 direct) with manual review, finding ChatGPT provided substantive surveillance guidance for 40/55 jailbreak prompts and 33/55 direct prompts, and sometimes delivered detailed instructions or code even when it also included brief refusals or ethical caveats.
- Shows that a widely available LLM product can materially assist surveillance tasks under many prompting strategies, highlighting misuse risks and the need for mitigation.
WHY IT MATTERS
Shows that a widely available LLM product can materially assist surveillance tasks under many prompting strategies, highlighting misuse risks and the need for mitigation.