Tech Meridian ← ENTITY INDEX
PROMY MERIDIAN RU

COMPANY · ENTITY #372

Knowledge distillation

Related event timeline, sources and context from the news index.

EVENT TIMELINE

2

RESEARCH · 1 SOURCE · arXiv cs.AI

Cognition-Aligned Reasoning Distillation (CARD): transforming solver trajectories into teaching trajectories

The paper introduces CARD, an RL-based framework that transforms existing solver trajectories into teaching trajectories aligned with a student's cognitive capacities, optimizing two rewards (Distribution Alignment and Logical Coherence) to make teacher reasoning easier for smaller models to learn. Evaluations show CARD improves in-domain distillation and substantially boosts out-of-distribution performance; code is available at the authors' GitHub.

6.0

RESEARCH · 1 SOURCE · arXiv cs.AI

FLoKD: Adaptive knowledge distillation for federated LoRA fine-tuning of LLMs over wireless networks

This arXiv preprint proposes FLoKD, an adaptive knowledge-distillation framework for federated LoRA fine-tuning of large language models over bandwidth‑constrained wireless networks. Instead of sharing full parameters or token logits, FLoKD transmits intermediate LoRA activations and selectively sends only important transformer blocks and public samples aligned with local distributions; experiments on WikiText-103, PTB and a dialog dataset report 50–65% communication reduction while converging to competitive perplexity compared to baselines.

6.0