Limits of Confidence in Diffusion
This paper analyzes discrete diffusion samplers (including remasking and uniform-state samplers) and proves that a diffusion step matches the training distribution only when the positions it writes are conditionally independent given already-fixed tokens. The authors show that products of per-position marginals cannot represent dependent groups, that identical per-position marginals can hide different joint dependencies, and validate these phenomena on the ScanAndAdd synthetic task where groups of two-or-more undetermined positions are dependent and the generated distribution exhibits 29× the sampling-noise-floor total variation despite per-sample metrics being 1.0.