Tech Meridian
← ENTITY INDEX
COMPANY · ENTITY #3414
Related event timeline, sources and context from the news index.
AWS published a guide and released a Ray LLM Deep Learning Container (DLC) that supports deploying a single large language model across multiple GPU nodes on Amazon EKS, using a disaggregated prefill/decode setup. The Ray LLM DLC packages vLLM, Ray Serve LLM, NIXL for KV-cache transfer, and the EFA stack, exposes an OpenAI‑compatible endpoint, and includes a helper (build_pd_openai_app) plus example code in a repository.
An Amazon post details an architecture that combines Amazon EKS, Elastic Fabric Adapter (EFA), and DeepEP to address the heterogeneous compute, high-bandwidth communication, and orchestration challenges of post-training MoE reinforcement learning (including RLHF and GRPO). The write-up explains balancing rollout generation and tightly coupled policy training, optimizing Expert Parallelism (EP) communication over EFA, and reports up to a 40% throughput improvement using DeepEP.
NVIDIA describes how to integrate its Resiliency Extension (NVRx) with PyTorch FSDP on Amazon EKS to reduce downtime for large multi-node GPU training. The post demonstrates async checkpointing (TorchAsyncCheckpoint), in-process restart, and an in-job restart launcher (ft_launcher), provides H100 2–8 node benchmarks, and publishes reproducible code and a pip-installable nvidia-resiliency-ext package.
Allow Google Analytics to measure visits? It stays off until you choose. You can change your choice at any time.