Agent4Sci testbed (arXiv:2610.28763v1) evaluates frontier coding agents on 20 ML‑for‑science benchmarks
The paper introduces Agent4Sci, a long‑horizon testbed that measures how well current coding agents can improve published best models across 20 scientific ML benchmarks in 11 domains. Empirically, frontier agents (e.g., Claude Opus) improve published baselines (Claude Opus: 10.4% mean relative improvement), but gains mainly come from ML‑engineering tricks; simple plug‑in skills for ideation and experiment management boost Claude Opus to 20.1% and Codex to 8.1%. Code and data are available on GitHub and Hugging Face.