Tech Meridian ← LIVE FEED
PROMY MERIDIAN RU

RELEASE · CODING · #3038

Google launches LiteRT Tensor API for building and optimizing on-device ML in C++

Google introduced the LiteRT Tensor API, a lightweight C++ API that lets developers build computation graphs operation-by-operation for on-device execution across CPU, GPU, and NPU backends, with features like direct weight ingestion (including .safetensors), zero-copy buffer interoperability, and compatibility with AI coding agents. Google demonstrated the API with a SAM 2 multi-object segmentation and tracking app and reported cross-platform benchmarks (e.g., about 1.5x faster than Apple MLX on a MacBook Pro in one test) and listed planned enhancements such as NPU-friendly ops and automatic full-graph quantization.

KEY POINTS

  1. Google introduced the LiteRT Tensor API, a lightweight C++ API that lets developers build computation graphs operation-by-operation for on-device execution across CPU, GPU, and NPU backends, with features like direct weight ingestion (including .safetensors), zero-copy buffer interoperability, and compatibility with AI coding agents.
  2. Google demonstrated the API with a SAM 2 multi-object segmentation and tracking app and reported cross-platform benchmarks (e.g., about 1.5x faster than Apple MLX on a MacBook Pro in one test) and listed planned enhancements such as NPU-friendly ops and automatic full-graph quantization.
  3. The API gives developers low-level, portable C++ graph control and performance features (direct weights, zero-copy buffers, agent compatibility) that can reduce platform-specific rewrites and speed up on-device ML development and execution.

WHY IT MATTERS

The API gives developers low-level, portable C++ graph control and performance features (direct weights, zero-copy buffers, agent compatibility) that can reduce platform-specific rewrites and speed up on-device ML development and execution.

SOURCES & TIMELINE

1