SciGame benchmark transforms literature into paired real and counterfactual scientific environments (arXiv:2610.37919v1)
SciGame is a controlled benchmark that converts scientific literature into structured "cards," each providing paired Original and dimensionally consistent Counterfactual environments and a shared task interface; it includes a Historical collection of canonical laws and a Temporal collection of papers published after model knowledge cutoffs. The benchmark evaluates agents in Identification and open-ended Discovery settings under Prior-only, Fixed, and Adaptive protocols; experiments on five frontier models find that familiar priors can hurt when relations change, adaptive experimentation helps stronger models but can harm weaker ones under tight budgets, and good explanations can mask quantitative failures, with exact symbolic recovery remaining the main bottleneck.