NVIDIA and Dao AI Lab publish ReplaySSM to speed SSM-based hybrid-model decoding
NVIDIA and the Dao AI Lab (Princeton) released ReplaySSM (paper on arXiv and code on GitHub), a technique that caches recent SSM inputs instead of full state to halve state memory traffic and speed SSM decoding kernels by ~1.46–1.80× (batch 256 on Blackwell Ultra B300). ReplaySSM also simplifies speculative-decoding rollback (1.85–2.10× higher speculative throughput on 120B-class models) and enables mixed-precision state (three of four tested models stay within ~2 percentage points of FP32 on GPQA‑Diamond); it's integrated into TensorRT‑LLM, vLLM, and FlashInfer and runs in production in GLM‑5.3‑Flash and Kimi‑K3.