Tech Meridian ← ENTITY INDEX
PROMY MERIDIAN RU

MODEL · ENTITY #18263

GPT-5.1

Related event timeline, sources and context from the news index.

EVENT TIMELINE

2

RESEARCH · 1 SOURCE · Hugging Face

AI2 releases 'Old Ideas, Novel Problems' benchmark showing LLM novelty judges are unstable

AI2 published a study and public benchmark, 'Old Ideas, Novel Problems', evaluating how reliably large language models judge the novelty of research ideas. Using an automated pipeline built on ICLR 2026 peer reviews, two test settings (Human-Only: 154 high-novel / 145 low-novel ideas; Human+Generated: 154 human ideas paired with 154 LLM-generated ideas), and six LLM judges (GPT-5.1/5.2/5.4 and Claude Sonnet/Opus variants), they find judgments are highly sensitive to small setup changes (a prompt tweak shifted accuracy by >50 percentage points), retrieval and extra reasoning do not consistently help, and two purpose-built systems underperformed a simple prompted Claude Opus 4.6 baseline.

7.0

RESEARCH · 1 SOURCE · WIRED AI

Thompson study: OpenAI GPT‑5 series and other models increasingly solve theoretical problems across sciences

A study by Neil Thompson with Gabriel Manso and Claudio Mayrink Verdun tested OpenAI models (o1, o3, GPT‑5, GPT‑5.1, GPT‑5.4, GPT‑5.5) on theory problems drawn from papers posted Dec 2025–May 2026; models were blocked from internet access and many ground‑truth solutions were published after the models' training cutoffs. The best models' success rate on problems in probability, statistics theory, economic theory, high‑energy theory, gravitation and cosmology, and quantum physics rose from roughly 2% in mid‑2024 to about 35% by mid‑2026.

8.0