AWS Lambda adds GPU inference on G‑series instances with up to 48 concurrent invocations sharing one model
AWS announced GPU support for Lambda Managed Instances, allowing functions deployed as OCI container images to run on EC2 G‑series instances (G4dn, G5, G6, G6f, Gr6, Gr6f, G6e). Functions declare GPU memory via a new AcceleratorConfig, up to 48 concurrent invocations can share a single model copy in GPU memory, AWS provides five GPU base images (CUDA, CuPy, PyTorch, vLLM, SGLang), and pricing is EC2 On‑Demand plus a 9.75% management fee and $0.20 per million requests; early hands‑on tests report strong throughput but tooling and quota frictions (Terraform support, GPU vCPU quotas) remain.