ReliabilityRoute paper: behavioral stress tests show when forecasting agents should reason
This arXiv paper treats retrieval, reasoning, market priors and historical analogs as observable agent behaviors in ForecastBench-style binary forecasting tasks and finds that which mechanism works best is source-dependent. The authors introduce ReliabilityRoute, a routing intervention that uses reliability features (historical coverage, market-prior availability and sharpness, evidence strength/disagreement, horizon) and show a fixed 2024-fitted rule closely matches a hand taxonomy while a walk-forward self-adjusting rule achieves the best mean Brier score among their deterministic systems across 16 later LLM vintages; gains are modest and historical/search baselines remain competitive; reproducibility artifacts are on GitHub.