Hierarchical agent distillation for LLMs (arXiv:2610.31918v1)
The paper introduces a hierarchical agent distillation framework that separates two levels of "on-policiness"—the policy that produces interaction histories and the policy that generates the token prefix of the current action—and lets practitioners control their pairing, mixture, and schedule. In a three-stage self-distillation study across three models and four interactive environments, the authors find that student-induced histories paired with teacher-generated action prefixes give the strongest standalone signal, fully on-policy data performs worst alone but helps in mixtures, and front-loading teacher contexts while delaying on-policy contexts improves results.