NVIDIA H100

2026-09-17

RESEARCH · 1 SOURCE · arXiv cs.AI

Inference Engineering Pareto Atlas (arXiv:2609.17863v1) maps cost/quality/latency trade-offs

The paper releases a Pareto atlas that maps cost, quality, and latency trade-offs for LLM inference by measuring 54 configurations of Qwen2.5-7B-Instruct on vLLM 0.12 across L4, A100, and H100 GPUs and calibrating a simulator that reproduces anchored measurements with <1.5% drift. Key findings: 18 of 36 calibrated configurations reach the Pareto frontier; combinations of optimizations outperform many single methods; AWQ 4bit cuts per-token latency to 0.34× but loses ~5.9% GSM8K accuracy; FP8 weights retain ~99.4% accuracy at 0.61–0.65× latency; a naive FP8 KV cache preserves throughput but answers none of 200 GSM8K questions correctly; H100 is best for tight latency while A100 is best for throughput and cost (~$0.106 per million tokens).

7.0

2026-09-16

CODING · 1 SOURCE · AWS Machine Learning

NVIDIA NVRx adds fault-tolerance to PyTorch FSDP training on Amazon EKS

NVIDIA describes how to integrate its Resiliency Extension (NVRx) with PyTorch FSDP on Amazon EKS to reduce downtime for large multi-node GPU training. The post demonstrates async checkpointing (TorchAsyncCheckpoint), in-process restart, and an in-job restart launcher (ft_launcher), provides H100 2–8 node benchmarks, and publishes reproducible code and a pip-installable nvidia-resiliency-ext package.

6.0

2026-09-16

MODELS · 1 SOURCE · Cohere

Cohere releases open-source "Transcribe Arabic" ASR model claiming best open-weight Arabic accuracy

Cohere published Transcribe Arabic, an open-source Arabic automatic speech recognition (ASR) model (Apache 2.0) based on its 2B ASR work. Cohere says the model achieves a 25.87 WER on the Hugging Face Arabic ASR leaderboard, outperforming Meta’s OmniASR-LLM-7B and OpenAI’s Whisper Large V3, and was preferred to Whisper in about 96% of human evaluation tests; weights are available on Hugging Face and via the Cohere API/Model Vault.

8.0

2026-09-16

CODING · 1 SOURCE · Cohere

Cohere publishes megakernel serving engine for North Mini Code with faster H100 decoding

Cohere describes a megakernel-based serving engine for its North Mini Code 30B model that runs BF16 on a single NVIDIA H100 and claims 1.25×–1.41× end-to-end speedup over vLLM, with a reported 292 tok/s (62% of SoL) at batch size 1 — about 1.58× faster than vLLM. The system supports production features (continuous batching, paged attention, ragged sequences), an OpenAI-compatible endpoint with tool calling, is implemented as a single CUDA file, and the code is available on GitHub.

7.0