MLX enables local and multi‑Mac inference of Muse Spark Open (816B) on Mac Studio
The MLX array framework added Apple Silicon support and demonstrates running frontier LLMs locally and in a distributed setup on Mac Studio. Using the latest MLX release and RDMA over Thunderbolt, a four‑node Mac Studio cluster runs Muse Spark Open (816B, MoE, NVFP4) at >36 tokens/s; smaller models (Qwen 3.8 and Mistral Medium 3.5) can fit on a single 512GB Mac Studio. Benchmarks with mlx_lm.benchmark report up to 4.6× faster time‑to‑first‑token and ~1.6× higher tokens/s on M5 Ultra versus M3 Ultra, and modest peak power (~200–300W) for inference.