RESEARCH · RESEARCH · #3696
Toward a Theory of Evaluability — formalizing evaluation when test data may be used in training
This paper formalizes 'evaluability' for machine learning models without assuming a held-out test set and studies when accurate black-box evaluation is possible if test samples might have been used during training. It fully characterizes distributions where evaluation from outputs alone is feasible (showing feasibility iff the distribution is close to small support), provides tight sample-complexity bounds, and shows that limited query access can expand evaluability via connections to self-correctors.
KEY POINTS
- This paper formalizes 'evaluability' for machine learning models without assuming a held-out test set and studies when accurate black-box evaluation is possible if test samples might have been used during training.
- It fully characterizes distributions where evaluation from outputs alone is feasible (showing feasibility iff the distribution is close to small support), provides tight sample-complexity bounds, and shows that limited query access can expand evaluability via connections to self-correctors.
- Helps determine when external evaluation or auditing of large foundation models is theoretically possible despite training-set contamination, guiding how evaluations and query access should be designed.
WHY IT MATTERS
Helps determine when external evaluation or auditing of large foundation models is theoretically possible despite training-set contamination, guiding how evaluations and query access should be designed.