Tech Meridian ← LIVE FEED
PROMY MERIDIAN RU

RESEARCH · RESEARCH · #3642

Paper evaluates MXFP6/MXFP8 quantization for R&R on AMD MI350X, reports measured GEMM speedups

arXiv:2611.17501v1 presents an empirical evaluation of MXFP8 and a native 6-bit MXFP6 format for industrial-scale recommendation & ranking (R&R) inference on the AMD MI350X. The authors report MXFP8 GEMM kernel speedups over FP16 of 1.33–1.55x and MXFP6 speedups of 1.73–1.99x (up to 2.90x on some layers), with end-to-end throughput gains from selective MXFP6 of 1.5–5.3% for one production ads ranking model (Model A) and 7.6–14.3% for a second (Model B), while respecting a stringent 0.01% Normalized Entropy (NE) parity target.

KEY POINTS

  1. arXiv:2611.17501v1 presents an empirical evaluation of MXFP8 and a native 6-bit MXFP6 format for industrial-scale recommendation & ranking (R&R) inference on the AMD MI350X.
  2. The authors report MXFP8 GEMM kernel speedups over FP16 of 1.33–1.55x and MXFP6 speedups of 1.73–1.99x (up to 2.90x on some layers), with end-to-end throughput gains from selective MXFP6 of 1.5–5.3% for one production ads ranking model (Model A) and 7.6–14.3% for a second (Model B), while respecting a stringent 0.01% Normalized Entropy (NE) parity target.
  3. If reproducible, native 6-bit MXFP6 that preserves strict R&R accuracy could enable materially lower-cost, higher-throughput production inference for ad-ranking systems and influence hardware/software quantization choices.

WHY IT MATTERS

If reproducible, native 6-bit MXFP6 that preserves strict R&R accuracy could enable materially lower-cost, higher-throughput production inference for ad-ranking systems and influence hardware/software quantization choices.

SOURCES & TIMELINE

1