RESEARCH · RESEARCH · #2546
DELA: controllable draft-level edits for diffusion language models (arXiv:2610.32096v1)
The paper introduces DELA (Diffusion Editing Language Adapter), a plug-in controller that enables explicit draft-level editing for frozen diffusion language models via four actions (Insert, Delete, Remask, Stop). DELA uses a two-phase decoding scheme—fast append-only drafting that preserves KV-cache reuse followed by global refinement—and a staged curriculum to train the controller; experiments on mathematical reasoning and code generation report up to 5.5 percentage points improvement over direct diffusion decoding.
KEY POINTS
- The paper introduces DELA (Diffusion Editing Language Adapter), a plug-in controller that enables explicit draft-level editing for frozen diffusion language models via four actions (Insert, Delete, Remask, Stop).
- DELA uses a two-phase decoding scheme—fast append-only drafting that preserves KV-cache reuse followed by global refinement—and a staged curriculum to train the controller; experiments on mathematical reasoning and code generation report up to 5.5 percentage points improvement over direct diffusion decoding.
- DELA separates high-level edit control from token-level denoising and enables efficient, targeted non-monotonic edits in DLMs while keeping the backbone frozen, which can improve controllability and efficiency for tasks like code and math generation.
WHY IT MATTERS
DELA separates high-level edit control from token-level denoising and enables efficient, targeted non-monotonic edits in DLMs while keeping the backbone frozen, which can improve controllability and efficiency for tasks like code and math generation.