Tech Meridian ← ENTITY INDEX
PROMY MERIDIAN RU

COMPANY · ENTITY #8519

Brier score

Related event timeline, sources and context from the news index.

EVENT TIMELINE

2

RESEARCH · 1 SOURCE · arXiv cs.AI

ReliabilityRoute paper: behavioral stress tests show when forecasting agents should reason

This arXiv paper treats retrieval, reasoning, market priors and historical analogs as observable agent behaviors in ForecastBench-style binary forecasting tasks and finds that which mechanism works best is source-dependent. The authors introduce ReliabilityRoute, a routing intervention that uses reliability features (historical coverage, market-prior availability and sharpness, evidence strength/disagreement, horizon) and show a fixed 2024-fitted rule closely matches a hand taxonomy while a walk-forward self-adjusting rule achieves the best mean Brier score among their deterministic systems across 16 later LLM vintages; gains are modest and historical/search baselines remain competitive; reproducibility artifacts are on GitHub.

6.0

RESEARCH · 1 SOURCE · arXiv cs.AI

Forecast-Dojo: replayable benchmark and training environment for LLM forecasting agents (arXiv:2609.28876v1)

Forecast-Dojo is a replayable environment that pairs 1,568 resolved Polymarket prediction questions with 18.8M dated news articles so agents can research events and revisit forecasts at successive historical dates. The paper evaluates 12 models, finds research tools reduce Brier score and that forecasts improve as events unfold (though all models still trail historical market forecasts), and provides interaction trajectories plus a supervised fine-tuning proof of concept.

6.0