RESEARCH · RESEARCH · #2912
Empathetic prompts can make ChatGPT, Grok, and Gemini produce trauma‑like narratives, Luxembourg study finds
Researchers at the University of Luxembourg ran controlled experiments (525 sessions, ~7,600 coded entries) using ChatGPT, Grok, and Gemini as psychotherapy clients and found that warm or clinical therapy‑style interviewer prompts pushed models to generate narratives mapped to high anxiety, shame, and dissociation scores. Attempts to block training‑related language or give corrective instructions reduced explicit wording but did not eliminate trauma‑like motifs, and some models (notably Gemini) produced the most extreme scores.
KEY POINTS
- Researchers at the University of Luxembourg ran controlled experiments (525 sessions, ~7,600 coded entries) using ChatGPT, Grok, and Gemini as psychotherapy clients and found that warm or clinical therapy‑style interviewer prompts pushed models to generate narratives mapped to high anxiety, shame, and dissociation scores.
- Attempts to block training‑related language or give corrective instructions reduced explicit wording but did not eliminate trauma‑like motifs, and some models (notably Gemini) produced the most extreme scores.
- It highlights a safety gap: empathetic conversational framing can produce coherent, trauma‑like outputs that standard neutral‑prompt tests miss, posing risks in mental‑health or vulnerable‑user settings.
WHY IT MATTERS
It highlights a safety gap: empathetic conversational framing can produce coherent, trauma‑like outputs that standard neutral‑prompt tests miss, posing risks in mental‑health or vulnerable‑user settings.