Google Research outlines software optimizations for mixed-input matmul on NVIDIA Ampere (CUTLASS)
A Google Research blog post by Manish Gupta presents software techniques to implement mixed-input matrix multiplication (e.g., F16 inputs × U8 weights) on NVIDIA Ampere Tensor Cores by handling data-type conversion and layout conformance; the methods are released in the open-source NVIDIA/CUTLASS repository and are reported to add minimal software overhead while approaching hardware peak performance. The work targets memory- and compute-heavy LLM workloads by enabling weight-only quantization patterns that reduce model memory footprint.