RESEARCH · RESEARCH · #1857
MagicBeanStalk (arXiv:2610.12659v1): recursive data synthesis to escape RLVR zero-advantage trap
The paper (arXiv:2610.12659v1) identifies a ‘zero-advantage’ dataset failure mode in RLVR—about 88% of problems in a large open-source math corpus are solved or fail in every rollout for a fixed target policy and budget—and shows filtering alone discards hard, informative problems. It introduces MagicBeanStalk, a top-down decomposition and bottom-up recursive data-synthesis method that grows seed problems into policy- and budget-calibrated families; continual RLVR on these problems raises gpt-oss-120b from 61.2 to 69.8 (+8.6) and DeepSeek v4Flash from 72.4 to 77.9 (+5.4) on 16K-token rollouts, and synthesizes 14K verifiable finance problems from 340 solutions improving SecQue and FinanceBench by 6.9 and 2.4 points respectively.
KEY POINTS
- The paper (arXiv:2610.12659v1) identifies a ‘zero-advantage’ dataset failure mode in RLVR—about 88% of problems in a large open-source math corpus are solved or fail in every rollout for a fixed target policy and budget—and shows filtering alone discards hard, informative problems.
- It introduces MagicBeanStalk, a top-down decomposition and bottom-up recursive data-synthesis method that grows seed problems into policy- and budget-calibrated families; continual RLVR on these problems raises gpt-oss-120b from 61.2 to 69.8 (+8.6) and DeepSeek v4Flash from 72.4 to 77.9 (+5.4) on 16K-token rollouts, and synthesizes 14K verifiable finance problems from 340 solutions improving SecQue and FinanceBench by 6.9 and 2.4 points respectively.
- Calibrated, recursive synthetic problems can provide much stronger, budget-aware training signal for RLVR than simply scaling or filtering corpora, producing large score gains on math and finance benchmarks.
WHY IT MATTERS
Calibrated, recursive synthetic problems can provide much stronger, budget-aware training signal for RLVR than simply scaling or filtering corpora, producing large score gains on math and finance benchmarks.