RESEARCH · RESEARCH · #526
OBC-Prune: outcome-based calibration improves pruning for large reasoning models (arXiv:2609.17890v1)
This paper proposes OBC-Prune, an outcome-based calibration method for pruning large reasoning models that builds difficulty-matched correct/incorrect rollouts and uses intervention-based analysis to score the causal importance of reasoning sentences. The per-token importance weights are used to rescale calibration activations for one-shot pruning methods (SparseGPT, Wanda, ALPS), and experiments on DeepSeek-R1-Distill-Qwen 1.5B/7B/14B at 40% and 50% sparsity report consistent improvements over calibration baselines on MATH500, LiveCodeBench, and AIME 2025.
KEY POINTS
- This paper proposes OBC-Prune, an outcome-based calibration method for pruning large reasoning models that builds difficulty-matched correct/incorrect rollouts and uses intervention-based analysis to score the causal importance of reasoning sentences.
- The per-token importance weights are used to rescale calibration activations for one-shot pruning methods (SparseGPT, Wanda, ALPS), and experiments on DeepSeek-R1-Distill-Qwen 1.5B/7B/14B at 40% and 50% sparsity report consistent improvements over calibration baselines on MATH500, LiveCodeBench, and AIME 2025.
- Preserving causally important reasoning circuits during pruning can reduce inference cost while maintaining or improving reasoning quality, making pruning more effective for chain-of-thought models.
WHY IT MATTERS
Preserving causally important reasoning circuits during pruning can reduce inference cost while maintaining or improving reasoning quality, making pruning more effective for chain-of-thought models.