Momento CEO Khawaja Shams presents 'Serving LLMs at Scale: The Hidden KV Cache Advantage' at QCon AI
Khawaja Shams (Co‑Founder & CEO of Momento) gave a QCon AI presentation describing distributed-systems approaches to scale LLM inference efficiently, focusing on GPU utilization, KV cache memory cliffs, tiered storage, tail-latency reduction, and disaggregated prefill/decode architectures. He framed these techniques as ways to improve fleet performance, cost economics, and availability for production LLM deployments.