RESEARCH · RESEARCH · #1251
SAGE: statistical acceptance gate reduces regressions in self‑evolving LLM agents
New arXiv paper (arXiv:2609.36043v1) introduces SAGE, a Statistical Acceptance Gate for self‑evolving LLM agents that (1) uses per‑item paired comparisons to detect regressions hidden by aggregate scores and (2) applies a one‑sided paired statistical test to accept edits only when wins are reliably greater than losses. Across five benchmarks and four backbone LLMs under an equal‑budget protocol, SAGE markedly lowers regression rates in 19 of 20 settings and matches or improves final scores (e.g., regression rate from 36.5% to 0% on LiveMath and from 42.8% to 0% on OfficeQA with DeepSeek‑V4).
KEY POINTS
- New arXiv paper (arXiv:2609.36043v1) introduces SAGE, a Statistical Acceptance Gate for self‑evolving LLM agents that (1) uses per‑item paired comparisons to detect regressions hidden by aggregate scores and (2) applies a one‑sided paired statistical test to accept edits only when wins are reliably greater than losses.
- Across five benchmarks and four backbone LLMs under an equal‑budget protocol, SAGE markedly lowers regression rates in 19 of 20 settings and matches or improves final scores (e.g., regression rate from 36.5% to 0% on LiveMath and from 42.8% to 0% on OfficeQA with DeepSeek‑V4).
- Prevents permanent regressions and mitigates Optimizer's Curse in self‑evolving agents by testing edits per item and requiring statistically reliable wins, improving robustness of autonomous model updates.
WHY IT MATTERS
Prevents permanent regressions and mitigates Optimizer's Curse in self‑evolving agents by testing edits per item and requiring statistically reliable wins, improving robustness of autonomous model updates.