Probe guidance: a method to steer diffusion language models
The paper introduces probe guidance, a method that uses frozen internal states from an existing diffusion model to produce a guidance signal without an extra forward pass at inference. Applied to continuous diffusion language models, probe guidance reportedly achieves new state-of-the-art unconditional generation and improves multiple-choice QA performance for a 1.7B DLM, and the authors analyze conditions under which traditional autoguidance works.