NVIDIA adds 'cuDF skills' to help AI agents rewrite Spark UDFs for GPU acceleration
NVIDIA describes a workflow and a new 'cuDF skills for Apache Spark' capability that uses agent-driven tooling to rewrite Spark UDFs into GPU-accelerable forms (via Spark SQL expressions, the RapidsUDF Java interface, or custom CUDA kernels), eliminating GPU fallback and achieving 22x+ speedups in tests. NVIDIA cites a case where a large consumer tech company converted 30+ production UDFs and reduced an operator's runtime from 274.55 to 12.78 hours on a 600‑billion‑row workload.