SPECTRUM: Proximal Spectral Modulation improves diversity retention in looped self-distillation for code generation
This arXiv paper introduces Looped Self-Distillation, a framework where a model repeatedly generates and learns from its own outputs under a fixed information budget, and presents SPECTRUM — a method that re-estimates loss-sensitive key/value geometry from a fixed reference and applies full-rank proximal spectral modulation. In experiments on MBPP, SPECTRUM preserved 89.9% of the initial model's 64-sample correct-AST richness after five rounds (vs. 66.4% for vanilla self-distillation), and the diversity improvement transferred to HumanEval+ and APPS Intro without additional recalibration.