RESEARCH · RESEARCH · #1849
Representation Geometry Regularization (RGR) mitigates capability degradation in LLM self-improvement
The paper (arXiv:2610.12855v1) identifies a failure mode in On-Policy Self-distillation (OPSD) where models lose previously acquired capabilities as training proceeds, linked to deterioration of hidden representation geometry (weaker intra-cluster compactness and inter-cluster separation). It proposes Representation Geometry Regularization (RGR), a plug-in post-training regularizer that preserves representation geometry; experiments across OPSD methods and benchmarks show RGR generally reduces capability degradation while keeping overall improvements, and code is released on GitHub.
KEY POINTS
- The paper (arXiv:2610.12855v1) identifies a failure mode in On-Policy Self-distillation (OPSD) where models lose previously acquired capabilities as training proceeds, linked to deterioration of hidden representation geometry (weaker intra-cluster compactness and inter-cluster separation).
- It proposes Representation Geometry Regularization (RGR), a plug-in post-training regularizer that preserves representation geometry; experiments across OPSD methods and benchmarks show RGR generally reduces capability degradation while keeping overall improvements, and code is released on GitHub.
- Addresses a concrete reliability issue in LLM self-improvement by preserving representation geometry, which can make OPSD methods more dependable for continued capability gains.
WHY IT MATTERS
Addresses a concrete reliability issue in LLM self-improvement by preserving representation geometry, which can make OPSD methods more dependable for continued capability gains.