Microsoft commits Maia 300 to multi-gigawatt inference deployments and touts tokens-per-watt gains

KEY POINTS

1. Microsoft says it will scale its Azure Maia family to multi-gigawatt deployments beginning in 2027 and make Maia the primary inference system for Microsoft services; Maia 200 is already in production for Copilot and MAI, with Maia 300 slated for production deployments in 2027.
2. The Maia design emphasizes HBM4 memory stacks, chiplet NUMA-aware placement, and a custom Ethernet-based accelerator fabric that enables peer-to-peer tensor traffic, while Microsoft claims up to ~40% better tokens-per-megawatt vs leading GPU systems—a metric that practitioners have questioned and that Microsoft has not fully defined or independently validated.
3. This shapes the next-generation cloud inference infrastructure — Microsoft’s architectural choices (memory-centric chiplets and a peer-to-peer fabric) and its disputed tokens-per-watt claims will influence benchmarking, supplier competition, and enterprise access to high-efficiency inference capacity.