NEWS · CODING · #2242
PyTorch 2.15 brings FP8, faster LLM decoding and new Metal MPS kernels for Macs
PyTorch 2.15 (Hugging Face contributors) improves the MPS backend on Apple Silicon: adds torch.float8_e4m3fn (FP8) support with F.scaled_mm, faster single‑row linear and grouped matmul paths that speed Mixture‑of‑Experts decoding 5–6.5× (e.g., Qwen3.6‑35B‑A3B from 9.4→61.3 tok/s on an M3 Ultra), an ~8× faster softmax for sampling, and many new Metal kernels that cut driver memory for some video decoders (Wan2.2) from ~270 GiB to ~8 GiB. The release also raises platform requirements to macOS 15+ and Python 3.11+, and previews further MPSGraph→Metal migrations in 2.16.
KEY POINTS
- PyTorch 2.15 (Hugging Face contributors) improves the MPS backend on Apple Silicon: adds torch.float8_e4m3fn (FP8) support with F.scaled_mm, faster single‑row linear and grouped matmul paths that speed Mixture‑of‑Experts decoding 5–6.5× (e.g., Qwen3.6‑35B‑A3B from 9.4→61.3 tok/s on an M3 Ultra), an ~8× faster softmax for sampling, and many new Metal kernels that cut driver memory for some video decoders (Wan2.2) from ~270 GiB to ~8 GiB.
- The release also raises platform requirements to macOS 15+ and Python 3.11+, and previews further MPSGraph→Metal migrations in 2.16.
- This materially improves the speed and memory footprint of running and training LLMs and media models on Macs, making practical local inference and smaller‑scale training much more feasible.
WHY IT MATTERS
This materially improves the speed and memory footprint of running and training LLMs and media models on Macs, making practical local inference and smaller‑scale training much more feasible.