NVIDIA Dynamo-Triton 26.07 enables TensorRT multi-device inference across GPUs
NVIDIA has enabled TensorRT multi-device inference in Dynamo-Triton release 26.07 (TensorRT multi-device is supported starting in TensorRT 11.0), allowing a single KIND_MODEL instance to own multiple GPUs and present one gRPC endpoint while using NCCL-backed distributed collectives. In a Cosmos 3 Nano video-generation demo, distributing a 44,160-token transformer across up to eight GPUs reduced end-to-end generation latency from 156.6s to 34.2s and produced a reported transformer RPC speedup of 6.09x; users can download Dynamo-Triton 26.07 from NGC and follow the TensorRT multi-device guide to reproduce the setup.