RESEARCH · RESEARCH · #3153
SemiAnalysis launches GDPx benchmark; top AI agent scores just 12%
Analytics firm SemiAnalysis released GDPx, a 206-task benchmark that runs AI agents in simulated multi-hour knowledge-work environments using professional software; Claude Opus 5.5 tops the leaderboard at 12%, followed by Gemini 4 Carbon (10.9%) and GPT-6 Astra (9.6%). Critics question the benchmark's secrecy, effects of provider safety filters, and unequal test conditions tied to each provider's agent tooling.
KEY POINTS
- Analytics firm SemiAnalysis released GDPx, a 206-task benchmark that runs AI agents in simulated multi-hour knowledge-work environments using professional software; Claude Opus 5.5 tops the leaderboard at 12%, followed by Gemini 4 Carbon (10.9%) and GPT-6 Astra (9.6%).
- Critics question the benchmark's secrecy, effects of provider safety filters, and unequal test conditions tied to each provider's agent tooling.
- GDPx attempts to measure agent performance in realistic, software-driven work settings and shows current agents perform poorly, while also sparking debate about benchmark design and comparability across providers.
WHY IT MATTERS
GDPx attempts to measure agent performance in realistic, software-driven work settings and shows current agents perform poorly, while also sparking debate about benchmark design and comparability across providers.