Google Research proposes 'early readouts' and a 'feature sieve' to mitigate spurious features and simplicity bias
Google Research authors Rishabh Tiwari and Pradeep Shenoy describe two interventions—using predictions from early intermediate layers ('early readouts') and inducing 'feature forgetting' via a 'feature sieve'—to detect and reduce reliance on spurious features and the simplicity bias in deep networks. They apply the early-readout signal to reweight teacher contributions during distillation and use a feature-sieving intervention to encourage models to find more predictive features, reporting improved worst-group accuracy across benchmarks (Waterbirds, CelebA, CivilComments, MNLI) and better generalization to unseen domains compared with prior methods.