DACA-GRPO: denoising-aware credit assignment improves GRPO for diffusion LLMs
The paper introduces DACA-GRPO, a lightweight plug-and-play enhancement for GRPO-style reinforcement learning on diffusion language models that addresses missing temporal credit assignment and mean-field likelihood bias. It adds Denoising Progress Scores (per-token importance weights from intermediate predictions) and Stratified Masking Likelihood (token strata to reduce mean-field bias), and reports consistent gains across seven benchmarks—up to 5.6 percentage points on math reasoning, 7.4 pp on code generation, 36.3 pp on constraint satisfaction, and 5.9 pp on JSON schema adherence.
Why it matters: By adding per-step credit assignment and reducing mean-field likelihood bias, DACA-GRPO materially improves RL optimization for diffusion LLMs, yielding substantial gains on reasoning and constrained-generation tasks.
OPEN EVENT →