Toward a Theory of Evaluability — formalizing evaluation when test data may be used in training
This paper formalizes 'evaluability' for machine learning models without assuming a held-out test set and studies when accurate black-box evaluation is possible if test samples might have been used during training. It fully characterizes distributions where evaluation from outputs alone is feasible (showing feasibility iff the distribution is close to small support), provides tight sample-complexity bounds, and shows that limited query access can expand evaluability via connections to self-correctors.