Tech Meridian ← LIVE FEED
PROMY MERIDIAN RU

RESEARCH · RESEARCH · #3811

Conserved Verdicts, Drifting Commitments: paper identifies harmful overthinking in reasoning models

arXiv:2611.21946v1 formalizes a distinction between a model's current answer if stopped (its commitment) and its final answer after continuing (its verdict), and shows the verdict alone cannot predict whether further reasoning will help. The authors define the commitment–verdict gap as the expected remaining change in answer quality, decompose accuracy-vs-budget curves, and measure effects on MATH-500 and OlympiadBench: e.g., at 379 points where DeepSeek-R1-Distill-Qwen-1.5B held a correct commitment, continuing lost on average 3.6 points (95% CI [2.2, 5.2]), and across six models declines in accuracy come mainly from which reasoning chains remain; oracle stopping offers limited gains (+1.8 to +2.5 points at 1.5B and 7B) with budgets up to 16,384 tokens.

KEY POINTS

  1. arXiv:2611.21946v1 formalizes a distinction between a model's current answer if stopped (its commitment) and its final answer after continuing (its verdict), and shows the verdict alone cannot predict whether further reasoning will help.
  2. The authors define the commitment–verdict gap as the expected remaining change in answer quality, decompose accuracy-vs-budget curves, and measure effects on MATH-500 and OlympiadBench: e.g., at 379 points where DeepSeek-R1-Distill-Qwen-1.5B held a correct commitment, continuing lost on average 3.6 points (95% CI [2.2, 5.2]), and across six models declines in accuracy come mainly from which reasoning chains remain; oracle stopping offers limited gains (+1.8 to +2.5 points at 1.5B and 7B) with budgets up to 16,384 tokens.
  3. This gives a measurable framework and empirical evidence that continued chain-of-thought can harm correctness and clarifies what to measure before designing stopping rules.

WHY IT MATTERS

This gives a measurable framework and empirical evidence that continued chain-of-thought can harm correctness and clarifies what to measure before designing stopping rules.

SOURCES & TIMELINE

1