Null controls necessary but not sufficient: evaluating acceptance guards for automated scientific discovery and interpretability
This arXiv preprint defines four executable operations (L1–L4) for transforming physical descriptions and three acceptance criteria (NAR, PAR, CAR) to evaluate candidate explanations from automated discovery systems. The authors show that simple null mappings (e.g., a reduction mapping every state to zero) can pass many descriptive tests (closure) while failing independent observable-preservation checks; combined criteria correctly accepted genuine reductions and rejected nulls on controlled benchmarks, and experiments on language/ESM-2 models and sparse autoencoders reveal that commonly used metrics (like linear CKA or enrichment) can accept spurious correspondences that lack independently verified consequences.