Apple engineers present scaling framework-agnostic AI agent infrastructure with Ray at QCon AI
At QCon AI, Apple engineers Deepak Mohanakumar Chandramouli and Bhumik Vinodkumar Thakkar described moving AI agents from local FastAPI prototypes to resilient production deployments on Kubernetes using Ray, covering RayCluster topology, stateful memory handling, autoscaling for dynamic traffic, control‑plane designs, multi‑region setups, safety guardrails, and an LLM‑as‑a‑judge evaluation pipeline. They illustrated these operational patterns with a trip‑planner agent example and shared production learnings and observability considerations.