PyTorch

2026-11-03

MODELS · 1 SOURCE · Hugging Face

Transformers' proposed MIC fixtures would have caught UMT5 future-token attention bug introduced with PyTorch SDPA

On July 30, 2026, Transformers switched T5 to use PyTorch's scaled dot-product attention (SDPA), and that change allowed the UMT5 decoder to attend to future tokens during training. The project describes Model Integration Contracts (transformers-mic): small, pinned fixtures plus runnable usage paths and recorded baselines run on cheap CPU runners; replaying MICs over commits shows the UMT5 fixture would have flagged the regression from its introduction until its fix on September 28, 2026.

7.0

2026-11-02

CODING · 1 SOURCE · InfoQ AI, ML & Data Engineering

Google and Meta open-source TorchTPU to provide native PyTorch backend for Google TPUs

Google, in collaboration with Meta, open-sourced TorchTPU, a native PyTorch backend for Google TPUs that prioritizes eager execution while also supporting compiled modes, distributed training, and hardware optimizations; migrating common PyTorch code generally requires only swapping cuda->tpu and nccl->tpu_dist or using torch.accelerator. Google validated TorchTPU across ~5,000 Hugging Face models and reported single-TPUv7 inference of 25.7k token/s (seq_len 5120) and 22.4k token/s (seq_len 10240) for a 30B Muse Glimmer model, plus multi-chip training numbers and large-scale MoE experiments up to 8,192 TPUv7 chips.

8.0

2026-10-29

RESEARCH · 1 SOURCE · NVIDIA Developer

TF32x3 and BF16x9 speed up Point Edge Transformer MLIP inference while keeping FP32-like accuracy

Evaluation on the Point Edge Transformer (PET) from Lab COSMO (EPFL) shows NVIDIA’s BF16x9 on Blackwell GPUs speeds large GEMMs by 2.2–2.5x (17,280-atom) and 1.2–1.4x (192-atom), while TF32x3 on Ampere speeds major GEMMs 1.7–1.9x and full MD steps up to 1.5x; TF32x3 preserves potential-energy smoothness, energy conservation in NVE trajectories, and converges structural relaxation (539-atom Cantor-alloy slab) in 218–222 steps with ~1.5x per-step speedup, whereas one-pass TF32 fails to reach force tolerance after 1,000 steps. The post notes how to enable BF16x9 via cuBLAS and points to the NVIDIA ALCHEMI Toolkit and the UPET repository for integration and model checkpoints.

7.0

2026-10-25

COMPANIES · 1 SOURCE · InfoQ AI, ML & Data Engineering

Google open-sources ML Drift GPU engine for on-device AI inference

Google released ML Drift, an Apache-2.0 open-source, cross-platform GPU compute engine for on-device AI and ML inference that abstracts multiple low-level APIs (OpenGL ES, OpenCL, Metal, WebGPU). ML Drift powers LiteRT, adds unified shaders via tensor virtualization, extensible custom ops, 5D tensor support, and hardware-specific optimizations for Arm, Intel, and Qualcomm GPUs; Google says it replaces the legacy TensorFlow Lite GPU delegate and reports lower memory overhead and app-level latency improvements in Chrome, YouTube Shorts, Photos, Adobe apps, and Snapchat.

8.0

2026-10-21

CODING · 1 SOURCE · Hugging Face

Profiling TorchNeuron on AWS Trainium — system-level walkthrough (Part 1)

This tutorial shows how to profile PyTorch workloads on AWS Trainium using TorchNeuron (torch-neuronx). It covers configuring torch.profiler with NeuronConfig and NeuronProfiler, exporting matched system/device traces (NEFF/NTFF), inspecting execution with Neuron Explorer, and compares three RMSNorm implementations (naive PyTorch ops, fused torch.rms_norm, and torch.compile) with reproducible examples in the torch-neuronx examples directory.

6.0

2026-10-20

MODELS · 1 SOURCE · AWS Machine Learning

i14 trains and serves Laplace 3B linear-attention model on AWS Trainium2 with TorchNeuron Native

i14 ported its Laplace 3B 3B-parameter linear-attention decoder (using Gated Linear Attention and a fixed 29.5 MB recurrent state) to AWS Trainium2 via TorchNeuron Native and native PyTorch operators. On a trn2.48xlarge instance the team reported ~120,000 training tokens/sec and serving throughput of 11,544 output tokens/sec across 1,280 concurrent sequences after replacing a Triton kernel, optimizing MuonFSDP2 collectives, and statically unrolling the GLA recurrence for the Trainium2 compiler.

7.0

2026-10-20

COMPANIES · 1 SOURCE · TechCrunch AI

Google releases open-source TorchTPU to run PyTorch on TPUs, challenging Nvidia

Google announced TorchTPU, an open-source PyTorch implementation that lets developers run existing PyTorch apps on Google’s TPUs with minimal changes; the move follows Amazon’s recent release of TorchNeuron for Trainium and accompanies efforts to decouple PyTorch from Nvidia-specific code. Google also released a TPU agent suite and updated LiteRT to assist porting and edge deployments.

8.0

2026-10-20

CODING · 1 SOURCE · Hugging Face

PyTorch 2.15 brings FP8, faster LLM decoding and new Metal MPS kernels for Macs

PyTorch 2.15 (Hugging Face contributors) improves the MPS backend on Apple Silicon: adds torch.float8_e4m3fn (FP8) support with F.scaled_mm, faster single‑row linear and grouped matmul paths that speed Mixture‑of‑Experts decoding 5–6.5× (e.g., Qwen3.6‑35B‑A3B from 9.4→61.3 tok/s on an M3 Ultra), an ~8× faster softmax for sampling, and many new Metal kernels that cut driver memory for some video decoders (Wan2.2) from ~270 GiB to ~8 GiB. The release also raises platform requirements to macOS 15+ and Python 3.11+, and previews further MPSGraph→Metal migrations in 2.16.

7.0

2026-10-20

CODING · 1 SOURCE · Hugging Face

Hugging Face launches TorchNeuron — native PyTorch backend for AWS Trainium (public beta Oct 20, 2026)

TorchNeuron is a new PyTorch plugin from Hugging Face that exposes Neuron as a native PyTorch device for AWS Trainium, enabling eager execution, torch.compile and native distributed APIs (DTensor, FSDP, TP). It brings Transformers, Diffusers and other Hugging Face libraries to Trainium without Optimum Neuron-specific forks, increases out-of-the-box text-generation model compatibility (89% of top 100 vs 46% with Optimum Neuron v0.4.5), and will be available as a public beta starting Oct 20, 2026 (pip install torch-neuronx).

8.0

2026-10-19

COMPANIES · 1 SOURCE · TechCrunch AI

AWS launches TorchNeuron to run PyTorch models on Trainium

Amazon Web Services released TorchNeuron, a PyTorch-compatible tool that AWS says lets developers run PyTorch apps on its Trainium AI chips with minimal configuration (one-line change). AWS also plans to contribute upstream changes to PyTorch to reduce framework tie‑ins to specific chips and argues the move gives startups easier access to AWS cloud and better price‑performance across training and inference.

8.0

2026-09-24

CODING · 1 SOURCE · NVIDIA Developer

Efficient MoE training for biological foundation models using NVIDIA Transformer Engine and MXFP8

NVIDIA published a BioNeMo MoE training recipe that uses Transformer Engine primitives—GroupedLinear, MXFP8 block-scaled 8-bit precision, and a fused GroupedMLP kernel (ForwardGroupedMLP_CuTeGEMMSwiGLU_MXFP8)—to reduce kernel-launch overhead, lower activation memory, and speed MoE training. In an eight‑GPU (NVIDIA B200) benchmark the recipe reached up to 2.21× throughput versus a Hugging Face baseline; the fused MXFP8 kernel requires NVIDIA Blackwell hardware.

6.0

2026-09-23

CODING · 1 SOURCE · Hugging Face

NVIDIA demonstrates using Warp and MJWarp to run MuJoCo at GPU scale (up to 2,048 envs)

NVIDIA describes migrating a MuJoCo workflow (an SO-101 follower arm) to MJWarp, an implementation of MuJoCo’s physics pipeline in NVIDIA Warp, and demonstrates advancing as many as 2,048 parallel environments on NVIDIA GPUs. The article explains Warp’s GPU kernel model, device arrays, composable launches and graph capture, and notes Warp features such as differentiable kernels and an opt-in deterministic mode (Warp ≥ 1.15); it focuses on preparing and scaling simulation environments rather than training policies.

6.0

2026-09-22

MODELS · 1 SOURCE · Hugging Face

Transformers adds GGUF (llama.cpp quant) support on main

Hugging Face's transformers (main branch) can now load and run GGUF quantized checkpoints using llama.cpp/ggml kernels, aiming for efficient local inference (initially optimized for Apple Silicon and Qwen3.5 models). The integration also enables serving GGUF checkpoints via transformers-serve and falls back to dequantization if compatible kernels are unavailable.

7.0

2026-09-16

CODING · 1 SOURCE · AWS Machine Learning

NVIDIA NVRx adds fault-tolerance to PyTorch FSDP training on Amazon EKS

NVIDIA describes how to integrate its Resiliency Extension (NVRx) with PyTorch FSDP on Amazon EKS to reduce downtime for large multi-node GPU training. The post demonstrates async checkpointing (TorchAsyncCheckpoint), in-process restart, and an in-job restart launcher (ft_launcher), provides H100 2–8 node benchmarks, and publishes reproducible code and a pip-installable nvidia-resiliency-ext package.

6.0

2024-03-06

RESEARCH · 1 SOURCE · Google Research

Croissant 1.0: a schema.org-based metadata format for ML-ready datasets, with support from major dataset hosts and frameworks

Croissant is a new ML-oriented metadata format (v1.0) built on schema.org that standardizes description and organization of datasets without changing underlying file formats. The release includes a spec, example datasets, an open-source Python validator/consumer/generator, a visual editor, and a Responsible AI (RAI) vocabulary extension, and is being adopted by Kaggle, Hugging Face, OpenML, indexed by Dataset Search, and made loadable in major frameworks via TensorFlow Datasets.

7.0