RESEARCH · RESEARCH · #2108
VISTA: value-informed schedule trimming for sparse SMC guidance in discrete diffusion (arXiv:2610.20669v1)
The paper introduces VISTA, a training-free method that selects a subset of timesteps to apply Sequential Monte Carlo (SMC) guidance in reward-guided discrete diffusion under a fixed compute budget. VISTA uses value statistics from a small number of full-guidance warmup generations to derive and optimize an empirical surrogate (via dynamic programming), yielding task-dependent sparse schedules that often match or approach full-guidance quality while substantially reducing per-generation latency (e.g., toxic-text: unique-toxic ratio 0.99 vs 1.00 and latency reduced from 204.23 s to 38.16 s, a 5.35× speedup).
KEY POINTS
- The paper introduces VISTA, a training-free method that selects a subset of timesteps to apply Sequential Monte Carlo (SMC) guidance in reward-guided discrete diffusion under a fixed compute budget.
- VISTA uses value statistics from a small number of full-guidance warmup generations to derive and optimize an empirical surrogate (via dynamic programming), yielding task-dependent sparse schedules that often match or approach full-guidance quality while substantially reducing per-generation latency (e.g., toxic-text: unique-toxic ratio 0.99 vs 1.00 and latency reduced from 204.23 s to 38.16 s, a 5.35× speedup).
- By optimizing which timesteps to guide, VISTA can cut SMC guidance compute dramatically while preserving alignment quality, making reward‑guided discrete diffusion more practical across modalities.
WHY IT MATTERS
By optimizing which timesteps to guide, VISTA can cut SMC guidance compute dramatically while preserving alignment quality, making reward‑guided discrete diffusion more practical across modalities.