RESEARCH · RESEARCH · #3642
Paper evaluates MXFP6/MXFP8 quantization for R&R on AMD MI350X, reports measured GEMM speedups
arXiv:2611.17501v1 presents an empirical evaluation of MXFP8 and a native 6-bit MXFP6 format for industrial-scale recommendation & ranking (R&R) inference on the AMD MI350X. The authors report MXFP8 GEMM kernel speedups over FP16 of 1.33–1.55x and MXFP6 speedups of 1.73–1.99x (up to 2.90x on some layers), with end-to-end throughput gains from selective MXFP6 of 1.5–5.3% for one production ads ranking model (Model A) and 7.6–14.3% for a second (Model B), while respecting a stringent 0.01% Normalized Entropy (NE) parity target.
KEY POINTS
- arXiv:2611.17501v1 presents an empirical evaluation of MXFP8 and a native 6-bit MXFP6 format for industrial-scale recommendation & ranking (R&R) inference on the AMD MI350X.
- The authors report MXFP8 GEMM kernel speedups over FP16 of 1.33–1.55x and MXFP6 speedups of 1.73–1.99x (up to 2.90x on some layers), with end-to-end throughput gains from selective MXFP6 of 1.5–5.3% for one production ads ranking model (Model A) and 7.6–14.3% for a second (Model B), while respecting a stringent 0.01% Normalized Entropy (NE) parity target.
- If reproducible, native 6-bit MXFP6 that preserves strict R&R accuracy could enable materially lower-cost, higher-throughput production inference for ad-ranking systems and influence hardware/software quantization choices.
WHY IT MATTERS
If reproducible, native 6-bit MXFP6 that preserves strict R&R accuracy could enable materially lower-cost, higher-throughput production inference for ad-ranking systems and influence hardware/software quantization choices.