NVIDIA releases AIPerf, a multiprocess LLM benchmarking tool
NVIDIA AIPerf replaces GenAI-Perf with a ground-up multiprocess architecture designed to avoid client-side bottlenecks for high-concurrency LLM inference benchmarking. It supports 15+ endpoint types, public datasets and trace replay formats (ShareGPT, Mooncake, Baseten, WEKA/AgentX), configurable arrival patterns, and reports TTFT, ITL, latency percentiles and GPU telemetry when DCGM or pynvml are available.