AlphaDiana: a measurement system for agents in verifiable reasoning tasks (arXiv:2610.20675v1)
AlphaDiana is a modular measurement system that links agent execution traces, resource records, and answer correctness while preserving harness-native control loops. The paper (arXiv:2610.20675v1) demonstrates macro analyses and micro interventions across three open-source models and three open-source harnesses on five answer-based benchmarks (and two interactive benchmarks for two models), finding that harnesses and component interventions (tools, skill guidance, retained memory) do not reliably improve accuracy and that execution/ token metrics must be interpreted alongside correctness.