Adaptive Reference Guidance (ARG) for prefix continuation in reasoning RL (arXiv:2610.11128v1)
The arXiv preprint (2610.11128v1) analyzes how much reference guidance to provide when training reasoning models via prefix continuation and derives a closed-form KL divergence for single continuations. From this analysis the authors propose Adaptive Reference Guidance (ARG), a learned prefix selector that constructs correct trajectories within a fixed generation budget and, when applied to GRPO with Qwen3-4B and Qwen3-8B across five mathematical reasoning benchmarks, achieves the highest aggregate pass@12 among evaluated methods with competitive sampled accuracy.