Measuring inference energy efficiency of 86 open-weight LLM checkpoints across benchmarks, reasoning modes, and hardware
An arXiv preprint (arXiv:2610.14389v1) reports direct measurements of inference energy for 86 open-weight model checkpoints across seven benchmarks and six hardware platforms. The study finds large energy-efficiency variation even among similarly capable models (most configurations use a median 2.6–4.8× more energy than a Pareto-optimal alternative, with up to 27×), reasoning increases energy by ~3.7× on average, generated-token count misorders 18–24% of model pairs as an energy proxy, FP8 yields ~25% energy savings with little score change, and changing hardware largely rescales energy without altering relative model rankings; the authors recommend reporting benchmark scores together with measured energy under a specified serving regime and output budget.