RESEARCH · RESEARCH · #2096
Temporal Specification Guidance (TSG) for controllable text-to-video generation
The paper introduces Temporal Specification Guidance (TSG), a training-free inference-time framework that encodes temporal requirements as Metric Temporal Logic specifications and turns them into differentiable satisfaction scores over video-text attention traces; gradients of these scores produce attention biases that steer denoising and augment classifier-free guidance. Evaluated on a 220-prompt benchmark spanning 11 temporal constraint categories, this instantiation improves specification-satisfaction accuracy by 9.5–20 percentage points across five text-to-video models, without external concept classifiers or backpropagation through the video transformer.
KEY POINTS
- The paper introduces Temporal Specification Guidance (TSG), a training-free inference-time framework that encodes temporal requirements as Metric Temporal Logic specifications and turns them into differentiable satisfaction scores over video-text attention traces; gradients of these scores produce attention biases that steer denoising and augment classifier-free guidance.
- Evaluated on a 220-prompt benchmark spanning 11 temporal constraint categories, this instantiation improves specification-satisfaction accuracy by 9.5–20 percentage points across five text-to-video models, without external concept classifiers or backpropagation through the video transformer.
- TSG provides a practical, model-agnostic way to enforce temporal ordering, timing, and persistence at inference time, improving controllability of text-to-video outputs without retraining models.
WHY IT MATTERS
TSG provides a practical, model-agnostic way to enforce temporal ordering, timing, and persistence at inference time, improving controllability of text-to-video outputs without retraining models.