Amazon SageMaker HyperPod Inference adds managed support for NVIDIA Dynamo
Amazon SageMaker HyperPod Inference now supports NVIDIA Dynamo via a HyperPod Inference add-on that installs Dynamo CRDs and serving/scheduling components, enabling multi-node and disaggregated LLM inference (separate prefill and decode workers) with EFA networking, coordinated GPU scheduling, and KV-aware routing. Customers can deploy via a bring-your-own DynamoGraphDeployment or an InferenceEndpointConfig with dynamoSpec, and choose engines such as vLLM, SGLang, or TensorRT-LLM on instances like ml.p5.48xlarge using AWS DLCs.