StateBench: benchmark for long-context state tracking in language models
StateBench is a new benchmark that measures how well language models track evolving entity state across long documents, organized into four difficulty levels (r-trivial, aperiodic, soluble, general finite) corresponding to increasingly complex kinds of variable assignment. The authors report that recurrent and hybrid architectures (e.g., Gated DeltaNet) generalize farther than transformers on these tasks, while leading frontier models including Gemini 3.8 Flash, Claude Opus 5.5, and GPT-6 Sol/Luna still break down on the hardest, long-dependency cases.