GUIDE · CODING · #2432
NVIDIA tutorial: use KvikIO to read data into GPU memory with GPUDirect Storage
NVIDIA published a tutorial demonstrating KvikIO—a high-level Python/C++ library that sits above the cuFile C API—to read data directly into GPU memory and verify whether the GPUDirect Storage (GDS) accelerated path was used. The guide explains KvikIO and cuFile path-selection behavior, relevant runtime settings (e.g., KVIKIO_COMPAT_MODE, KVIKIO_GDS_THRESHOLD, cufile.json), how to collect cuFile stats (CUFILE_STATS_LEVEL=3) and inspect gds_stats, and was tested on an x86_64 system with an NVIDIA H100, CUDA 13.3 and cuFile 1.18.
KEY POINTS
- NVIDIA published a tutorial demonstrating KvikIO—a high-level Python/C++ library that sits above the cuFile C API—to read data directly into GPU memory and verify whether the GPUDirect Storage (GDS) accelerated path was used.
- The guide explains KvikIO and cuFile path-selection behavior, relevant runtime settings (e.g., KVIKIO_COMPAT_MODE, KVIKIO_GDS_THRESHOLD, cufile.json), how to collect cuFile stats (CUFILE_STATS_LEVEL=3) and inspect gds_stats, and was tested on an x86_64 system with an NVIDIA H100, CUDA 13.3 and cuFile 1.18.
- Efficiently loading large datasets directly into GPU memory reduces CPU staging and helps keep accelerators fed for large-scale AI training and inference workloads.
WHY IT MATTERS
Efficiently loading large datasets directly into GPU memory reduces CPU staging and helps keep accelerators fed for large-scale AI training and inference workloads.