RESEARCH · RESEARCH · #2542
MPQ: message-passing approach to post-training quantization (arXiv:2610.32127v1)
This arXiv preprint proposes MPQ (Message-Passing Quantization), which frames row-wise post-training quantization as posterior inference over discrete codebooks and preserves soft posterior information across weight blocks using vector approximate message passing before committing to hard codewords via block coordinate descent. The authors report improved perplexity and zero-shot accuracy versus hard-decision baselines (e.g., GPTQ/LDLQ) on 2-bit lattice and 3–4-bit scalar quantization for dense LLMs from the Llama, Qwen, and Mistral families, and provide a reference implementation on GitHub.
KEY POINTS
- This arXiv preprint proposes MPQ (Message-Passing Quantization), which frames row-wise post-training quantization as posterior inference over discrete codebooks and preserves soft posterior information across weight blocks using vector approximate message passing before committing to hard codewords via block coordinate descent.
- The authors report improved perplexity and zero-shot accuracy versus hard-decision baselines (e.g., GPTQ/LDLQ) on 2-bit lattice and 3–4-bit scalar quantization for dense LLMs from the Llama, Qwen, and Mistral families, and provide a reference implementation on GitHub.
- PTQ is critical for deploying large language models, and MPQ's soft-posterior message-passing approach reduces quantization degradation on low-bit weights, potentially improving real-world LLM efficiency and accuracy.
WHY IT MATTERS
PTQ is critical for deploying large language models, and MPQ's soft-posterior message-passing approach reduces quantization degradation on low-bit weights, potentially improving real-world LLM efficiency and accuracy.