RESEARCH · RESEARCH · #1848
CToM: train LLMs to anticipate user responses and self-revise
This arXiv paper proposes Collaborative Theory of Mind (CToM), a framework that trains LLMs to draft a response, predict the user's reply to that draft, and use the predicted reply to guide internal revision before presenting an answer. Combined with group-relative policy optimization (GRPO) and a prediction-based loss, CToM improves collaborative task performance and interaction efficiency across mathematical reasoning, coding, and writing — e.g., raising final accuracy on a collaborative math task from 74.1% to 79.6% versus naive GRPO and reducing user replies by 8% on coding and 17% on writing under equal reward.
KEY POINTS
- This arXiv paper proposes Collaborative Theory of Mind (CToM), a framework that trains LLMs to draft a response, predict the user's reply to that draft, and use the predicted reply to guide internal revision before presenting an answer.
- Combined with group-relative policy optimization (GRPO) and a prediction-based loss, CToM improves collaborative task performance and interaction efficiency across mathematical reasoning, coding, and writing — e.g., raising final accuracy on a collaborative math task from 74.1% to 79.6% versus naive GRPO and reducing user replies by 8% on coding and 17% on writing under equal reward.
- Anticipating user replies and self-revision directly targets common failures in multi-turn collaboration, improving both final task accuracy and the number of interaction turns needed.
WHY IT MATTERS
Anticipating user replies and self-revision directly targets common failures in multi-turn collaboration, improving both final task accuracy and the number of interaction turns needed.