NEWS · MODELS · #2728
Artificial Analysis overhauls Intelligence Index v5; Kimi K3.1 Pro falls to ninth
Artificial Analysis released a revamped Intelligence Index (v5) that uses 60% private test sets, widening the score gap between open and closed models to 12 points (from 3). The update dropped Kimi K3.1 Pro from fourth to ninth and also cut Grok 4.7’s score by 12 points; the AA team revised ambiguous items in its AA-Omniscience knowledge test for the new index.
KEY POINTS
- Artificial Analysis released a revamped Intelligence Index (v5) that uses 60% private test sets, widening the score gap between open and closed models to 12 points (from 3).
- The update dropped Kimi K3.1 Pro from fourth to ninth and also cut Grok 4.7’s score by 12 points; the AA team revised ambiguous items in its AA-Omniscience knowledge test for the new index.
- Benchmark methodology changes (more private tests) materially affect model rankings and visibility, revealing that previous public benchmarks may have overstated some open models’ performance.
WHY IT MATTERS
Benchmark methodology changes (more private tests) materially affect model rankings and visibility, revealing that previous public benchmarks may have overstated some open models’ performance.