NVIDIA outlines NVFP4, Vera Rubin platform, and five co-design levers to boost AI throughput and KV-cache efficiency
In a blog post, NVIDIA says it is using five co-designed levers — the 4-bit NVFP4 numerical format, wide expert parallelism in TensorRT-LLM, Dynamo (fleet-aware KV-cache routing and the KV Cache Runner), the Vera CPU with Rubin interconnect, and a CMX pod-level storage tier — to free HBM, raise arithmetic intensity, and improve throughput for agentic inference. NVIDIA reports NVFP4 reduces model footprint roughly 3.5x versus FP16 (1.8x versus FP8), NVFP4 KV cache halves cache footprint versus FP8, and claims the Vera Rubin NVL72 can deliver up to 30x the throughput per megawatt of GB300 NVL72 on an agentic benchmark.