RESEARCH · RESEARCH · #2473
D-Skill: distilling black-box teacher agents into natural-language skills
D-Skill is a framework that analyzes paired teacher-student rollouts to extract interpretable natural-language skill documents that are provided to the student at inference time. Built entirely offline from fixed trajectories with no further environment interaction or weight updates, D-Skill closes 75%–95% of the teacher-student performance gap across SpreadsheetBench-Verified, ALFWorld, and chess puzzles, outperforms one-epoch SFT baselines after a single offline pass, remains competitive with multi-epoch SFT, and yields 5.7x–8.3x lower inference cost per task for the student under the same model family.
KEY POINTS
- D-Skill is a framework that analyzes paired teacher-student rollouts to extract interpretable natural-language skill documents that are provided to the student at inference time.
- Built entirely offline from fixed trajectories with no further environment interaction or weight updates, D-Skill closes 75%–95% of the teacher-student performance gap across SpreadsheetBench-Verified, ALFWorld, and chess puzzles, outperforms one-epoch SFT baselines after a single offline pass, remains competitive with multi-epoch SFT, and yields 5.7x–8.3x lower inference cost per task for the student under the same model family.
- Because it enables interpretable, low-cost capability transfer from black-box teacher agents without weight updates, offering a practical alternative to traditional weight-based distillation.
WHY IT MATTERS
Because it enables interpretable, low-cost capability transfer from black-box teacher agents without weight updates, offering a practical alternative to traditional weight-based distillation.