NEWS · COMPANIES · #4032
AWS Lambda adds GPU inference on G‑series instances with up to 48 concurrent invocations sharing one model
AWS announced GPU support for Lambda Managed Instances, allowing functions deployed as OCI container images to run on EC2 G‑series instances (G4dn, G5, G6, G6f, Gr6, Gr6f, G6e). Functions declare GPU memory via a new AcceleratorConfig, up to 48 concurrent invocations can share a single model copy in GPU memory, AWS provides five GPU base images (CUDA, CuPy, PyTorch, vLLM, SGLang), and pricing is EC2 On‑Demand plus a 9.75% management fee and $0.20 per million requests; early hands‑on tests report strong throughput but tooling and quota frictions (Terraform support, GPU vCPU quotas) remain.
KEY POINTS
- AWS announced GPU support for Lambda Managed Instances, allowing functions deployed as OCI container images to run on EC2 G‑series instances (G4dn, G5, G6, G6f, Gr6, Gr6f, G6e).
- Functions declare GPU memory via a new AcceleratorConfig, up to 48 concurrent invocations can share a single model copy in GPU memory, AWS provides five GPU base images (CUDA, CuPy, PyTorch, vLLM, SGLang), and pricing is EC2 On‑Demand plus a 9.75% management fee and $0.20 per million requests; early hands‑on tests report strong throughput but tooling and quota frictions (Terraform support, GPU vCPU quotas) remain.
- This brings serverless GPU inference into the native AWS event ecosystem, simplifying deployment of hosted models and enabling denser concurrency on GPU instances.
WHY IT MATTERS
This brings serverless GPU inference into the native AWS event ecosystem, simplifying deployment of hosted models and enabling denser concurrency on GPU instances.