Tech Meridian ← LIVE FEED
PROMY MERIDIAN RU

RESEARCH · RESEARCH · #3572

Termination lag in coding agents (arXiv:2611.14960v1)

The paper (arXiv:2611.14960v1) defines and measures “termination lag” — steps between an agent's first correct patch and the end of its run — by recording and testing every distinct repository patch across five models, two agent frameworks, and three SWE-bench datasets. It finds 38–82% of successful runs produce a correct solution on the initial edit, yet agents often continue working: median post-solution steps account for 35–37% for Qwen3-Coder and 54–60% for Laguna-XS.2 on SWE-bench Verified and Multilingual, and in 1–8% of cases agents later undo a previously correct patch.

KEY POINTS

  1. The paper (arXiv:2611.14960v1) defines and measures “termination lag” — steps between an agent's first correct patch and the end of its run — by recording and testing every distinct repository patch across five models, two agent frameworks, and three SWE-bench datasets.
  2. It finds 38–82% of successful runs produce a correct solution on the initial edit, yet agents often continue working: median post-solution steps account for 35–37% for Qwen3-Coder and 54–60% for Laguna-XS.2 on SWE-bench Verified and Multilingual, and in 1–8% of cases agents later undo a previously correct patch.
  3. This quantifies a common inefficiency and occasional regression risk in code-writing agents, affecting evaluation, compute cost, and reliability.

WHY IT MATTERS

This quantifies a common inefficiency and occasional regression risk in code-writing agents, affecting evaluation, compute cost, and reliability.

SOURCES & TIMELINE

1