Tech Meridian
← ENTITY INDEX
COMPANY · ENTITY #1730
Related event timeline, sources and context from the news index.
This arXiv paper (2610.41112v1) evaluates three calibration approaches—HiGPTQ, HiAutoRound, and HiFlatQuant—for the HiFloat4 (HiF4) 4-bit block floating-point format. The authors report that these methods narrow the accuracy gap to BF16 to under 1% and outperform NVFP4 across a range of models and downstream tasks compared with direct HiF4 conversion.
Olmo has released Olmo-core 3, a redesigned open mixture-of-experts (MoE) training framework intended to scale MoE models into the trillion-parameter range while preserving computational efficiency. The stack replaces an FSDP-based implementation with a DDP-centered system, combines expert/pipeline parallelism and other optimizations, reports a 2.7× throughput uplift on a 47B-parameter MoE across eight NVIDIA B300 GPUs, and shows MXFP8 delivering about 21% higher throughput than BF16 in controlled tests.
Cohere published Transcribe Arabic, an open-source Arabic automatic speech recognition (ASR) model (Apache 2.0) based on its 2B ASR work. Cohere says the model achieves a 25.87 WER on the Hugging Face Arabic ASR leaderboard, outperforming Meta’s OmniASR-LLM-7B and OpenAI’s Whisper Large V3, and was preferred to Whisper in about 96% of human evaluation tests; weights are available on Hugging Face and via the Cohere API/Model Vault.
Cohere describes a megakernel-based serving engine for its North Mini Code 30B model that runs BF16 on a single NVIDIA H100 and claims 1.25×–1.41× end-to-end speedup over vLLM, with a reported 292 tok/s (62% of SoL) at batch size 1 — about 1.58× faster than vLLM. The system supports production features (continuous batching, paged attention, ragged sequences), an OpenAI-compatible endpoint with tool calling, is implemented as a single CUDA file, and the code is available on GitHub.
A Google Research blog post by Manish Gupta presents software techniques to implement mixed-input matrix multiplication (e.g., F16 inputs × U8 weights) on NVIDIA Ampere Tensor Cores by handling data-type conversion and layout conformance; the methods are released in the open-source NVIDIA/CUTLASS repository and are reported to add minimal software overhead while approaching hardware peak performance. The work targets memory- and compute-heavy LLM workloads by enabling weight-only quantization patterns that reduce model memory footprint.
Allow Google Analytics to measure visits? It stays off until you choose. You can change your choice at any time.