ACED-Bench: neuro-symbolic benchmark to evaluate how LLM agents acquire and act on causal evidence
The paper introduces ACED-BENCH, a neuro-symbolic benchmark that separates natural-language scenario text from hidden probabilistic graphical models to test whether LLM-based agents can design experiments, interpret finite evidence, and make causal decisions. The main leaderboard ACED-CORE contains 180 tasks (60 set-valued ancestry/descendancy questions and 120 constrained causal-decision tasks) and ACED-QUAL is a 150-question yes/no qualification; the authors evaluate seven contemporary and two earlier LLMs across three active agent designs and use an evidence-ledger analysis to localize investigation failures.