NEWS · RESEARCH · #292
Google Research proposes 'early readouts' and a 'feature sieve' to mitigate spurious features and simplicity bias
Google Research authors Rishabh Tiwari and Pradeep Shenoy describe two interventions—using predictions from early intermediate layers ('early readouts') and inducing 'feature forgetting' via a 'feature sieve'—to detect and reduce reliance on spurious features and the simplicity bias in deep networks. They apply the early-readout signal to reweight teacher contributions during distillation and use a feature-sieving intervention to encourage models to find more predictive features, reporting improved worst-group accuracy across benchmarks (Waterbirds, CelebA, CivilComments, MNLI) and better generalization to unseen domains compared with prior methods.
KEY POINTS
- Google Research authors Rishabh Tiwari and Pradeep Shenoy describe two interventions—using predictions from early intermediate layers ('early readouts') and inducing 'feature forgetting' via a 'feature sieve'—to detect and reduce reliance on spurious features and the simplicity bias in deep networks.
- They apply the early-readout signal to reweight teacher contributions during distillation and use a feature-sieving intervention to encourage models to find more predictive features, reporting improved worst-group accuracy across benchmarks (Waterbirds, CelebA, CivilComments, MNLI) and better generalization to unseen domains compared with prior methods.
- This matters because the methods offer automated diagnostics and practical interventions to reduce spurious-feature reliance and simplicity bias, improving worst‑group robustness and cross‑domain generalization in common benchmark tasks.
WHY IT MATTERS
This matters because the methods offer automated diagnostics and practical interventions to reduce spurious-feature reliance and simplicity bias, improving worst‑group robustness and cross‑domain generalization in common benchmark tasks.