GAUSS: workload-aware analytical and Monte‑Carlo simulator for LLM inference serving
GAUSS is a workload-aware simulator (arXiv:2610.37989v1) that predicts full distributions of time-to-first-token (TTFT) and time-between-tokens (TBT) plus sustainable request rates for LLM inference. It couples phase-specific latency surfaces for prefill and decode with a Lindley-recurrence queueing model and uses ensemble Monte‑Carlo sampling to capture tail latencies and evaluate operating points, routing, and scheduling policies without relying on single-trace replay.