AutoData: agentic search discovers improved pre-training data selection (arXiv:2609.19754v1)
The arXiv preprint introduces AutoData, an agent that searches a program space of executable selection algorithms (scoring, stratification, stochastic rules) to optimize pre-training data selection using validation feedback from a proxy model. In an overnight search AutoData found a recipe that outperforms existing human-designed curation pipelines and transfers to larger scales, improving the downstream CORE metric.