Taste-Bench benchmark shows AI agents struggle with long-term decision forks
Researchers at Microsoft and City University of Hong Kong released Taste-Bench, a 502-question benchmark that evaluates an agent's 'taste'—the ability to choose between branching long-term paths by freezing 'decision forks' in agent trajectories. The best model tested (GPT-5.6 Sol) averaged 59.7% accuracy, most models scored far lower, and a distilled student from Qwen3.6-27B improved judgment accuracy from 30% to 47.9% and raised agent success on 41 held-out SWE-bench Pro tasks from 14.6% to 33.7%.