RESEARCH · RESEARCH · #1994
DeepMind says AI-driven self-improvement remains far off despite automation gains
DeepMind engineer Haydn Belfield reports no measurable acceleration of AI research driven by AI itself, based on a General Capability Index covering 123 models and 92 benchmarks; the team finds Gemini 4 Argon solves about half of 74 RE-Bench ML engineering tasks (vs. 27% for Gemini 3.7 Flash) and rates the risk from AI-driven research automation as currently low. Belfield also notes AI writes over 75% of production code but 88% of code changes still require human oversight and every change is reviewed by a human before submission.
KEY POINTS
- DeepMind engineer Haydn Belfield reports no measurable acceleration of AI research driven by AI itself, based on a General Capability Index covering 123 models and 92 benchmarks; the team finds Gemini 4 Argon solves about half of 74 RE-Bench ML engineering tasks (vs.
- 27% for Gemini 3.7 Flash) and rates the risk from AI-driven research automation as currently low.
- Belfield also notes AI writes over 75% of production code but 88% of code changes still require human oversight and every change is reviewed by a human before submission.
WHY IT MATTERS
Provides empirical measurements that counter claims of imminent AI-driven 'research acceleration' and informs risk assessments and policy discussions about automation of scientific work.