Amazon SageMaker AI adds automatic monitoring and recommendations for inference endpoints
Amazon SageMaker AI now monitors generative-AI inference endpoints (default for vLLM serving containers on GPU instances) and flags latency drift, unused GPU capacity, and missing serving optimizations. Findings are sent via AWS Health and open an agentic experience in SageMaker Studio where teams can run benchmarked inference recommendations on separate compute and apply changes manually; the monitoring inspects operational signals only and does not read requests or responses.