RESEARCH · RESEARCH · #2028
Correction Training for Diffusion Models (arXiv:2610.14136v2)
The paper introduces a learned "corrector" that is trained on the base denoiser's own predictions to close the training–inference gap caused by exposure bias in iterative diffusion sampling. At each denoising step the corrector iteratively refines the denoiser's clean-sample prediction; evaluated on DiT, SiT, and JiT this approach substantially improves FID for unguided and unconditional generation, is competitive with classifier-free guidance (CFG), can be combined with CFG to improve diversity, and its inference cost can be removed by distilling refined predictions back into the base model.
KEY POINTS
- The paper introduces a learned "corrector" that is trained on the base denoiser's own predictions to close the training–inference gap caused by exposure bias in iterative diffusion sampling.
- At each denoising step the corrector iteratively refines the denoiser's clean-sample prediction; evaluated on DiT, SiT, and JiT this approach substantially improves FID for unguided and unconditional generation, is competitive with classifier-free guidance (CFG), can be combined with CFG to improve diversity, and its inference cost can be removed by distilling refined predictions back into the base model.
- Addresses exposure bias in diffusion sampling and yields practical quality gains (and a distillation path to remove extra inference cost) for unguided/unconditional generation.
WHY IT MATTERS
Addresses exposure bias in diffusion sampling and yields practical quality gains (and a distillation path to remove extra inference cost) for unguided/unconditional generation.