Not established from the available sources.
OPENAI · MODEL RELEASE TRACKER
GPT-5.4
GPT-5.4 is a specific GPT-series model from OpenAI. OpenAI noted that GPT-5.4 exhibited low controllability of its internal chain-of-thought (it “could barely manipulate [its] inner monologue on command”), a property relevant to monitoring and safety.CURRENT SNAPSHOT1/5 DIMENSIONS WITH DATA
The dimensions that change the decision.
Not established from the available sources.
Not established from the available sources.
ALFWorld success improvement with EvoBeliefIn EvoBelief experiments on ALFWorld (without actor-side transition memory), EvoBelief improved task success by 31.3% relative to the matched Frozen World Model when paired with GPT-5.4.
Inclusion in studyGPT-5.4 was one of the OpenAI models evaluated by the study (listed alongside o1, o3, GPT-5, GPT-5.1, and GPT-5.5).
Scientific fields testedThe study evaluated models on theoretical problems drawn from probability, statistics theory, economic theory, high-energy theory, gravitation and cosmology, and quantum physics.
Internet access during testingResearchers blocked internet access during testing, preventing the models from looking up solutions online.
Agreement with benchmark reference judge (HealthBench Professional)In the study, GPT-5.4 is the benchmark's reference judge on HealthBench Professional; the decision model Jev matched GPT-5.4 on 91% of criteria.
Evaluated in novelty benchmarkIncluded among six LLM judges (GPT-5.1, GPT-5.2, GPT-5.4, Claude Sonnet 4.5, Claude Opus 4.5, and Claude Opus 4.6) in the "Old Ideas, Novel Problems" benchmark for testing LLMs' ability to evaluate research idea novelty.
Effect of small prompt changes on accuracyIn one experiment reported in the benchmark, changing just a few words in a prompt shifted a model's accuracy in identifying which of two ideas is more novel by more than 50 percentage points.
Not established from the available sources.
VERIFIABLE FACTS
Every value stays attached to a source and date.
BENCHMARKS · ALFWorld success improvement with EvoBeliefINDEPENDENTLY SUPPORTED
In EvoBelief experiments on ALFWorld (without actor-side transition memory), EvoBelief improved task success by 31.3% relative to the matched Frozen World Model when paired with GPT-5.4.
CAPABILITIES · Used as a frozen language model in EvoBelief experimentsINDEPENDENTLY SUPPORTED
GPT-5.4 was used as a frozen language model in the EvoBelief study's experiments.
BENCHMARKS · Inclusion in studyDEVELOPER CLAIM
GPT-5.4 was one of the OpenAI models evaluated by the study (listed alongside o1, o3, GPT-5, GPT-5.1, and GPT-5.5).
BENCHMARKS · Scientific fields testedDEVELOPER CLAIM
The study evaluated models on theoretical problems drawn from probability, statistics theory, economic theory, high-energy theory, gravitation and cosmology, and quantum physics.
BENCHMARKS · Internet access during testingDEVELOPER CLAIM
Researchers blocked internet access during testing, preventing the models from looking up solutions online.
LIMITATIONS · Training cutoff relative to ground-truth papersDEVELOPER CLAIM
The solutions used as ground truth were published after the training cutoff for GPT-5.5 and preceding models, implying GPT-5.4 could not have been trained on those solutions.
BENCHMARKS · Agreement with benchmark reference judge (HealthBench Professional)INDEPENDENTLY SUPPORTED
In the study, GPT-5.4 is the benchmark's reference judge on HealthBench Professional; the decision model Jev matched GPT-5.4 on 91% of criteria.
BENCHMARKS · Evaluated in novelty benchmarkDEVELOPER CLAIM
Included among six LLM judges (GPT-5.1, GPT-5.2, GPT-5.4, Claude Sonnet 4.5, Claude Opus 4.5, and Claude Opus 4.6) in the "Old Ideas, Novel Problems" benchmark for testing LLMs' ability to evaluate research idea novelty.
LIMITATIONS · Sensitivity to evaluation setupDEVELOPER CLAIM
The study found that novelty evaluation by LLMs, including GPT-5.4, is far from reliable: seemingly small changes to the evaluation setup can produce dramatically different judgments of the same ideas.
BENCHMARKS · Effect of small prompt changes on accuracyDEVELOPER CLAIM
In one experiment reported in the benchmark, changing just a few words in a prompt shifted a model's accuracy in identifying which of two ideas is more novel by more than 50 percentage points.
SAFETY · Chain-of-thought (inner monologue) controllabilityDEVELOPER CLAIM
OpenAI reported that GPT-5.4 "could barely manipulate their inner monologue on command," indicating low controllability of chains of thought and potential monitoring limitations.
WHAT CHANGED
Stored passport versions, without reconstructed history.
SAFETY · Chain-of-thought (inner monologue) controllability+ OpenAI reported that GPT-5.4 "could barely manipulate their inner monologue on command," indicating low controllability of chains of thought and potential monitoring limitations.
BENCHMARKS · Evaluated in novelty benchmark+ Included among six LLM judges (GPT-5.1, GPT-5.2, GPT-5.4, Claude Sonnet 4.5, Claude Opus 4.5, and Claude Opus 4.6) in the "Old Ideas, Novel Problems" benchmark for testing LLMs' ability to evaluate research idea novelty.
BENCHMARKS · Effect of small prompt changes on accuracy+ In one experiment reported in the benchmark, changing just a few words in a prompt shifted a model's accuracy in identifying which of two ideas is more novel by more than 50 percentage points.
LIMITATIONS · Sensitivity to evaluation setup+ The study found that novelty evaluation by LLMs, including GPT-5.4, is far from reliable: seemingly small changes to the evaluation setup can produce dramatically different judgments of the same ideas.
BENCHMARKS · Agreement with benchmark reference judge (HealthBench Professional)+ In the study, GPT-5.4 is the benchmark's reference judge on HealthBench Professional; the decision model Jev matched GPT-5.4 on 91% of criteria.
BENCHMARKS · Scientific fields tested+ The study evaluated models on theoretical problems drawn from probability, statistics theory, economic theory, high-energy theory, gravitation and cosmology, and quantum physics.
BENCHMARKS · Inclusion in study+ GPT-5.4 was one of the OpenAI models evaluated by the study (listed alongside o1, o3, GPT-5, GPT-5.1, and GPT-5.5).
BENCHMARKS · Internet access during testing+ Researchers blocked internet access during testing, preventing the models from looking up solutions online.
LIMITATIONS · Training cutoff relative to ground-truth papers+ The solutions used as ground truth were published after the training cutoff for GPT-5.5 and preceding models, implying GPT-5.4 could not have been trained on those solutions.