RESEARCH · RESEARCH · #2692
Cognition-Aligned Reasoning Distillation (CARD): transforming solver trajectories into teaching trajectories
The paper introduces CARD, an RL-based framework that transforms existing solver trajectories into teaching trajectories aligned with a student's cognitive capacities, optimizing two rewards (Distribution Alignment and Logical Coherence) to make teacher reasoning easier for smaller models to learn. Evaluations show CARD improves in-domain distillation and substantially boosts out-of-distribution performance; code is available at the authors' GitHub.
KEY POINTS
- The paper introduces CARD, an RL-based framework that transforms existing solver trajectories into teaching trajectories aligned with a student's cognitive capacities, optimizing two rewards (Distribution Alignment and Logical Coherence) to make teacher reasoning easier for smaller models to learn.
- Evaluations show CARD improves in-domain distillation and substantially boosts out-of-distribution performance; code is available at the authors' GitHub.
- Aligning teacher trajectories to student cognition can make reasoning distillation more sample-efficient and improve robustness to distribution shift, addressing a practical mismatch between large solvers (often RLHF-trained) and smaller student models.
WHY IT MATTERS
Aligning teacher trajectories to student cognition can make reasoning distillation more sample-efficient and improve robustness to distribution shift, addressing a practical mismatch between large solvers (often RLHF-trained) and smaller student models.