Tech Meridian ← ENTITY INDEX
PROMY MERIDIAN RU

COMPANY · ENTITY #10937

MATH-500

Related event timeline, sources and context from the news index.

EVENT TIMELINE

5

RESEARCH · 1 SOURCE · arXiv cs.AI

Conserved Verdicts, Drifting Commitments: paper identifies harmful overthinking in reasoning models

arXiv:2611.21946v1 formalizes a distinction between a model's current answer if stopped (its commitment) and its final answer after continuing (its verdict), and shows the verdict alone cannot predict whether further reasoning will help. The authors define the commitment–verdict gap as the expected remaining change in answer quality, decompose accuracy-vs-budget curves, and measure effects on MATH-500 and OlympiadBench: e.g., at 379 points where DeepSeek-R1-Distill-Qwen-1.5B held a correct commitment, continuing lost on average 3.6 points (95% CI [2.2, 5.2]), and across six models declines in accuracy come mainly from which reasoning chains remain; oracle stopping offers limited gains (+1.8 to +2.5 points at 1.5B and 7B) with budgets up to 16,384 tokens.

7.0

RESEARCH · 1 SOURCE · arXiv cs.AI

Paper evaluates certify-or-decline proof-step verification for open-weight mathematical reasoning

This arXiv v1 paper studies 'certify-or-decline' proof-step verification applied to open-weight LLMs on GPQA, AIME-2025, and a MATH-500 subset. Certification substantially raises answer-key-match precision (e.g., +31 points on GPQA strong config, +46 on AIME) while answering far fewer attempts (35% and 21% respectively); on near-ceiling MATH-500 gains are ≤5 points with 13–27 point coverage loss. An output-token-matched self-consistency selector attains 82.9% precision at 35% coverage versus the verifier's pooled 87.6%; the authors identify a frequent failure mode where many problems end in invalid proof construction before reaching a judge, and external-LLM acceptance of rendered certified chains (61.0–67.6%) is much lower than the verifier's answer-key match (87.6%). The audits lack human or formal-kernel ground truth and open-weight auditor pairs show near-zero chance-corrected agreement.

6.0

RESEARCH · 1 SOURCE · arXiv cs.AI

Mixture-of-Agents paper measures per-token 'sufficient compute' and shows routing/drafting gains

arXiv:2610.02491v1 introduces a Mixture-of-Agents (MoA) methodology that measures each token's upper-bound 'sufficient compute' by testing which of 15 progressively larger models can reproduce a reference token; a 0.5B agent reproduces 92–95% of tokens, while the costliest 10% of tokens account for an estimated 64–80% of FLOPs. Using the MoA-derived map, model routing and drafting strategies reduce projected latency (e.g., from 7.59s to 5.12s on MATH-500) and cut draft-token use and projected latency versus fixed baselines, indicating remaining headroom for smarter allocation controllers.

7.0

RESEARCH · 1 SOURCE · arXiv cs.AI

GoldiMask: context selection and target weighting for fine-tuning diffusion language models

arXiv:2609.38385v1 introduces GoldiMask, a supervised fine-tuning procedure for discrete diffusion language models that selects which tokens to reveal as context via an approximate submodular objective and weights remaining prediction targets by their benefit and learnability. Across three backbones and three datasets the paper reports higher average accuracy in most settings (including reasoning and code generation) and reduced decoding iterations on GSM8K and MATH-500 under confidence-threshold parallel decoding; ablations show both context selection and target weighting contribute to the gains.

7.0

RESEARCH · 1 SOURCE · arXiv cs.AI

Separating coverage from specialization in LLM harnesses (arXiv:2609.35873v1)

The paper introduces a controlled evaluation that disentangles answer coverage, repeatable task advantages, and pre-execution selection for generated LLM harnesses. On 386 MATH-500 tasks, comparisons of eight generated harnesses against a baseline with nine identical copies show that repeated identical programs produce 2.16 percentage points of repeat-averaged oracle headroom; generated programs display repeatable score patterns that mainly reveal persistent weaknesses rather than useful persistent wins, the frozen selector yields 0.00 percentage-point gain, and both populations reach 98.70% oracle coverage at 27 harness executions; BIRD traces locate failures in mechanism implementation, activation, and output validity, leading the authors to argue that coverage and repeatability alone do not justify claims of useful specialization and to propose stricter evaluation standards for harness diversity.

5.0