FLoKD: Adaptive knowledge distillation for federated LoRA fine-tuning of LLMs over wireless networks
This arXiv preprint proposes FLoKD, an adaptive knowledge-distillation framework for federated LoRA fine-tuning of large language models over bandwidth‑constrained wireless networks. Instead of sharing full parameters or token logits, FLoKD transmits intermediate LoRA activations and selectively sends only important transformer blocks and public samples aligned with local distributions; experiments on WikiText-103, PTB and a dialog dataset report 50–65% communication reduction while converging to competitive perplexity compared to baselines.