Tech Meridian ← ENTITY INDEX
PROMY MERIDIAN RU

TOPIC · ENTITY #35206

test-set contamination

Related event timeline, sources and context from the news index.

EVENT TIMELINE

1

RESEARCH · 1 SOURCE · Apple Machine Learning Research

Toward a Theory of Evaluability — formalizing evaluation when test data may be used in training

This paper formalizes 'evaluability' for machine learning models without assuming a held-out test set and studies when accurate black-box evaluation is possible if test samples might have been used during training. It fully characterizes distributions where evaluation from outputs alone is feasible (showing feasibility iff the distribution is close to small support), provides tight sample-complexity bounds, and shows that limited query access can expand evaluability via connections to self-correctors.

7.0