RESEARCH · RESEARCH · #2258
ALTK-Evolve (GPT-4.1) uses learned guidelines to top τ²-bench retail leaderboard without weight updates
ALTK-Evolve extracts short behavioral guidelines from agent trajectories using GPT-4.1 and appends them to the agent's system prompt; with no model weight updates it achieves 67.5% Pass^4 on τ²-bench retail (outperforming every retail entry in the Oct 12, 2026 leaderboard snapshot) and improves mean accuracy and Pass^4 in both retail and airline domains. The method uses GPT-4.1 as both the guideline extractor and agent (with GPT-5.5 simulating customers), compares guideline-support thresholds (sup3 vs lossless), and reports up to +10 percentage-point gains in Pass^4 versus baseline.
KEY POINTS
- ALTK-Evolve extracts short behavioral guidelines from agent trajectories using GPT-4.1 and appends them to the agent's system prompt; with no model weight updates it achieves 67.5% Pass^4 on τ²-bench retail (outperforming every retail entry in the Oct 12, 2026 leaderboard snapshot) and improves mean accuracy and Pass^4 in both retail and airline domains.
- The method uses GPT-4.1 as both the guideline extractor and agent (with GPT-5.5 simulating customers), compares guideline-support thresholds (sup3 vs lossless), and reports up to +10 percentage-point gains in Pass^4 versus baseline.
- This shows that extracting and appending compact, high-support behavioral rules to prompts can materially improve agent reliability on constrained customer-service benchmarks without retraining models, which matters for deployable, low-cost agent improvements.
WHY IT MATTERS
This shows that extracting and appending compact, high-support behavioral rules to prompts can materially improve agent reliability on constrained customer-service benchmarks without retraining models, which matters for deployable, low-cost agent improvements.