RegretBench and BackUpper: measuring and automating backups for coding agents
An arXiv paper (arXiv:2610.28934v1) introduces RegretBench, a 205-task benchmark of real-world "regret" moments where users wished they'd preserved state, and BackUpper, a harness plug-in that checks tool calls and preserves affected state before modifications. Across six frontier models agents complete 86.1% of tasks but back up before regret moments only 15.2%; BackUpper covers all regret actions, restores states with 92.2% average similarity, costs about $0.014 per task, uses 7× fewer tokens than a self-backup baseline, and can match user backup preferences with an 84.5% F1 score.