Google launches LiteRT Tensor API for building and optimizing on-device ML in C++
Google introduced the LiteRT Tensor API, a lightweight C++ API that lets developers build computation graphs operation-by-operation for on-device execution across CPU, GPU, and NPU backends, with features like direct weight ingestion (including .safetensors), zero-copy buffer interoperability, and compatibility with AI coding agents. Google demonstrated the API with a SAM 2 multi-object segmentation and tracking app and reported cross-platform benchmarks (e.g., about 1.5x faster than Apple MLX on a MacBook Pro in one test) and listed planned enhancements such as NPU-friendly ops and automatic full-graph quantization.