RESEARCH · RESEARCH · #1990
Thompson study: OpenAI GPT‑5 series and other models increasingly solve theoretical problems across sciences
A study by Neil Thompson with Gabriel Manso and Claudio Mayrink Verdun tested OpenAI models (o1, o3, GPT‑5, GPT‑5.1, GPT‑5.4, GPT‑5.5) on theory problems drawn from papers posted Dec 2025–May 2026; models were blocked from internet access and many ground‑truth solutions were published after the models' training cutoffs. The best models' success rate on problems in probability, statistics theory, economic theory, high‑energy theory, gravitation and cosmology, and quantum physics rose from roughly 2% in mid‑2024 to about 35% by mid‑2026.
KEY POINTS
- A study by Neil Thompson with Gabriel Manso and Claudio Mayrink Verdun tested OpenAI models (o1, o3, GPT‑5, GPT‑5.1, GPT‑5.4, GPT‑5.5) on theory problems drawn from papers posted Dec 2025–May 2026; models were blocked from internet access and many ground‑truth solutions were published after the models' training cutoffs.
- The best models' success rate on problems in probability, statistics theory, economic theory, high‑energy theory, gravitation and cosmology, and quantum physics rose from roughly 2% in mid‑2024 to about 35% by mid‑2026.
- Rapid gains in models' ability to solve theory problems imply AI could materially change how theoretical science is done, while creating new verification and authorship challenges.
WHY IT MATTERS
Rapid gains in models' ability to solve theory problems imply AI could materially change how theoretical science is done, while creating new verification and authorship challenges.