RESEARCH · RESEARCH · #3861
Preview: CatBoost predictor forecasts fine-tuning wall-clock time far better than FLOP estimates
A preview of a forthcoming paper analyzes 272 supervised fine-tuning jobs (≈7,382 GPU-hours) run on NVIDIA H200 GPUs and finds that a CatBoost regressor using only pre-dispatch configuration features achieves R²=0.98, MAE=1.71 h (median error 0.51 h), while a FLOP-based analytic estimate explains far less (R²=0.31, MAE=12.41 h). The learned model leverages features that expose overheads (evaluation cadence, checkpoint I/O, per-GPU token volumes, etc.), making it usable for scheduling on shared clusters without runtime logs.
KEY POINTS
- A preview of a forthcoming paper analyzes 272 supervised fine-tuning jobs (≈7,382 GPU-hours) run on NVIDIA H200 GPUs and finds that a CatBoost regressor using only pre-dispatch configuration features achieves R²=0.98, MAE=1.71 h (median error 0.51 h), while a FLOP-based analytic estimate explains far less (R²=0.31, MAE=12.41 h).
- The learned model leverages features that expose overheads (evaluation cadence, checkpoint I/O, per-GPU token volumes, etc.), making it usable for scheduling on shared clusters without runtime logs.
- More accurate pre-dispatch duration estimates let shared GPU clusters schedule jobs with less idle time and fewer delayed experiments, improving utilization and delivery predictability.
WHY IT MATTERS
More accurate pre-dispatch duration estimates let shared GPU clusters schedule jobs with less idle time and fewer delayed experiments, improving utilization and delivery predictability.