LRMoE: demand-side load recalibration for memory-aware MoE training
A new arXiv paper (arXiv:2610.37720v1) introduces LRMoE, a memory-aware training system for sparse Mixture-of-Experts models that jointly plans retention quotas and recalibrates backward demand to reduce communication, computation, and buffer use on memory-constrained GPUs. Evaluated on Qwen3-30B-A3B and GLM4.5-106B-A12B, LRMoE reportedly cuts cumulative MoE-layer training time to fixed-loss targets by 21.2–44.8% versus the SYMI baseline.