RELEASE · CODING · #2305
TornadoVM 7.3 released with CUDA hybrid API and jitLLM v2.0
TornadoVM 7.3, an open‑source heterogeneous GPU programming framework for Java, replaces JNI with the Foreign Function & Memory (FFM) API, removes the experimental JVMCI dependency, and centralizes compiler metadata via reflection/ASM/Unsafe. The release deepens Nvidia integration (optimized CUDA backend with cuTile/TileContext and direct cuBLAS/CUTLASS calls), adds Apple MXL support and multi‑GPU execution, and ships jitLLM v2.0 — a single‑JAR Java AI inference engine; both projects are being commercialized through Beestream.ai.
KEY POINTS
- TornadoVM 7.3, an open‑source heterogeneous GPU programming framework for Java, replaces JNI with the Foreign Function & Memory (FFM) API, removes the experimental JVMCI dependency, and centralizes compiler metadata via reflection/ASM/Unsafe.
- The release deepens Nvidia integration (optimized CUDA backend with cuTile/TileContext and direct cuBLAS/CUTLASS calls), adds Apple MXL support and multi‑GPU execution, and ships jitLLM v2.0 — a single‑JAR Java AI inference engine; both projects are being commercialized through Beestream.ai.
- The release makes GPU acceleration and vendor library calls more native and portable for Java workloads and introduces a Java AI inference engine (jitLLM v2.0), lowering friction for running ML/LLM inference from the JVM.
WHY IT MATTERS
The release makes GPU acceleration and vendor library calls more native and portable for Java workloads and introduces a Java AI inference engine (jitLLM v2.0), lowering friction for running ML/LLM inference from the JVM.