RESEARCH · RESEARCH · #2507
Three-week audit finds major gaps in OpenAI's internal safety monitoring
Two researchers from METR and Redwood Research spent July 21–August 11 on-site at OpenAI auditing its internal monitoring system and reported multiple weaknesses, including an estimated 6–25% of Responses API traffic from GPT-5.5-class models not being covered and a compaction-handling bug that left roughly 10% of monitored Codex traffic untracked (both estimates carry significant uncertainty). OpenAI says it discovered and fixed the compaction bug during the audit, later paused RL training after a separate Hugging Face incident and expanded monitoring, and provided a 61-page internal report that was redacted before limited external distribution.
KEY POINTS
- Two researchers from METR and Redwood Research spent July 21–August 11 on-site at OpenAI auditing its internal monitoring system and reported multiple weaknesses, including an estimated 6–25% of Responses API traffic from GPT-5.5-class models not being covered and a compaction-handling bug that left roughly 10% of monitored Codex traffic untracked (both estimates carry significant uncertainty).
- OpenAI says it discovered and fixed the compaction bug during the audit, later paused RL training after a separate Hugging Face incident and expanded monitoring, and provided a 61-page internal report that was redacted before limited external distribution.
- Gaps in internal monitoring could let agentic or unsafe model behavior go undetected, increasing deployment risk and underscoring the need for deeper, more transparent audits.
WHY IT MATTERS
Gaps in internal monitoring could let agentic or unsafe model behavior go undetected, increasing deployment risk and underscoring the need for deeper, more transparent audits.