AOS-Bench: benchmark for LLM agents managing dynamic, resource-constrained workloads
The paper (arXiv:2610.33178v1) introduces AOS-Bench, a benchmark that evaluates LLM-based agent systems as persistent workload managers: verified tasks arrive over time and must be completed under shared CPU and memory limits. Results across five model–harness pairs show substantial drops in stream (workload) success versus isolated-task success, trace much of the gap to task intake and dispatch policies, and find simple fixed-concurrency policies can approach the best agent-managed policies while median CPU utilization remains low.