RESEARCH · RESEARCH · #1112
TGL-NSGA-II: teacher-guided low-fidelity NAS for TinyML (arXiv:2609.30553v1)
arXiv:2609.30553v1 introduces TGL-NSGA-II, a low-fidelity framework for constrained TinyML neural architecture search that uses a pretrained teacher to stratify samples and a short KD-Lite distillation plus a Gaussian-process surrogate to score candidates. The paper reports improved proxy ranking (Kendall-τ 0.74 and 0.62 on keyword spotting and bird-call tasks), a 41% reduction in proxy-score variance versus random evaluation, better multi-objective search metrics under a fixed budget, and 2.2x faster runs than full NSGA-II, while noting the guarantees apply to population-level low-fidelity evaluation and do not prove full evolutionary convergence.
KEY POINTS
- arXiv:2609.30553v1 introduces TGL-NSGA-II, a low-fidelity framework for constrained TinyML neural architecture search that uses a pretrained teacher to stratify samples and a short KD-Lite distillation plus a Gaussian-process surrogate to score candidates.
- The paper reports improved proxy ranking (Kendall-τ 0.74 and 0.62 on keyword spotting and bird-call tasks), a 41% reduction in proxy-score variance versus random evaluation, better multi-objective search metrics under a fixed budget, and 2.2x faster runs than full NSGA-II, while noting the guarantees apply to population-level low-fidelity evaluation and do not prove full evolutionary convergence.
- Provides a theoretically-analyzed, teacher-guided low-fidelity evaluation method that improves ranking reliability and speeds up constrained evolutionary NAS for TinyML under limited evaluation budgets.
WHY IT MATTERS
Provides a theoretically-analyzed, teacher-guided low-fidelity evaluation method that improves ranking reliability and speeds up constrained evolutionary NAS for TinyML under limited evaluation budgets.