RESEARCH · RESEARCH · #2110
RealtimeBench benchmark and TORSI method evaluate LLM agents under a live clock
arXiv:2610.20665v1 introduces RealtimeBench, a 180‑task benchmark across six game types that evaluates language‑model agents while environments continue running during inference, and proposes TORSI (Two‑Timescale Offline Recursive Self‑Improvement). The paper reports that GPT‑6.1‑sol outperforms a faster model (Jev) when games are paused but falls behind when games run continuously; applying TORSI—where a slow teacher produces reusable instructions for a fast student and validation gates updates—raises Jev’s average test score from 13.51 to 24.19 and transfers improvements to three other models without retuning.
KEY POINTS
- arXiv:2610.20665v1 introduces RealtimeBench, a 180‑task benchmark across six game types that evaluates language‑model agents while environments continue running during inference, and proposes TORSI (Two‑Timescale Offline Recursive Self‑Improvement).
- The paper reports that GPT‑6.1‑sol outperforms a faster model (Jev) when games are paused but falls behind when games run continuously; applying TORSI—where a slow teacher produces reusable instructions for a fast student and validation gates updates—raises Jev’s average test score from 13.51 to 24.19 and transfers improvements to three other models without retuning.
- Real‑time benchmarks and the TORSI two‑timescale training approach address a key gap—agents must balance deliberation and fast action to stay effective in continuously changing environments.
WHY IT MATTERS
Real‑time benchmarks and the TORSI two‑timescale training approach address a key gap—agents must balance deliberation and fast action to stay effective in continuously changing environments.