Tech Meridian ← ENTITY INDEX
PROMY MERIDIAN RU

COMPANY · ENTITY #10829

GPQA

Related event timeline, sources and context from the news index.

EVENT TIMELINE

2

RESEARCH · 1 SOURCE · arXiv cs.AI

Paper evaluates certify-or-decline proof-step verification for open-weight mathematical reasoning

This arXiv v1 paper studies 'certify-or-decline' proof-step verification applied to open-weight LLMs on GPQA, AIME-2025, and a MATH-500 subset. Certification substantially raises answer-key-match precision (e.g., +31 points on GPQA strong config, +46 on AIME) while answering far fewer attempts (35% and 21% respectively); on near-ceiling MATH-500 gains are ≤5 points with 13–27 point coverage loss. An output-token-matched self-consistency selector attains 82.9% precision at 35% coverage versus the verifier's pooled 87.6%; the authors identify a frequent failure mode where many problems end in invalid proof construction before reaching a judge, and external-LLM acceptance of rendered certified chains (61.0–67.6%) is much lower than the verifier's answer-key match (87.6%). The audits lack human or formal-kernel ground truth and open-weight auditor pairs show near-zero chance-corrected agreement.

6.0

RESEARCH · 1 SOURCE · arXiv cs.AI

arXiv paper: generate-transform decomposition explains when small-LLM team scaling helps across orchestration architectures

This arXiv preprint (arXiv:2609.36104v1) evaluates eight agent orchestration architectures across five instruction-tuned 7–9B models and several benchmarks (GSM8K, GSMHard, ARC, GPQA, MMLU, and an executable-code task) up to 30 calls. The authors introduce an exact generate–transform decomposition that splits accuracy change into coverage and transformation effects, finding that returns to adding agents are sharply task- and architecture-dependent: Proposer-Critic scales steeply and yields large gains on arithmetic problems (up to +17 points) but not on multiple-choice benchmarks, and token cost per budget still varies ~2.1×.

7.0