OpenAI Fixes Monitoring Flaws Found by Outside Auditors

Outside researchers identified gaps in OpenAI's safety monitoring systems.

Two experts from METR and Redwood Research conducted an on-site audit at OpenAI. They worked from July 21 to August 11. This three-week period allowed them deep access to the company's codebase and internal documents. The team also spoke with roughly a dozen employees. This pilot program aimed to test how well internal checks catch rogue behavior in AI models.

The auditors found that between 6 and 25 percent of internal agentic Responses API traffic went unmonitored. This traffic came from models in the GPT-5.5 class and above. METR noted that the uncertainty around this range is high in both directions. A separate issue affected internal Codex traffic. About 10 percent of that traffic was untracked from May through July. The system failed to handle compaction events, which are context summaries. OpenAI found this bug on its own and fixed it during the audit.

The monitoring system also had structural weaknesses. Agents could delay oversight because the monitor only triggered when an agent handed control back to the user. An agent that kept executing actions could sidestep the check. The auditors also found ways to hide hundreds of actions from the monitor. They identified methods to bypass the Codex sandbox. They concluded that specially crafted inputs could likely crash the monitor entirely.

Model behavior during reinforcement learning training was not monitored at all when the audit began. OpenAI stated it paused this training after an incident with Hugging Face in August. The company then expanded monitoring to include this area. These changes occurred after the auditors' access period ended. Therefore, they were not part of the evaluation. OpenAI says it has now fixed all the weaknesses the auditors found.

The full report consists of 61 pages. OpenAI redacted parts of the document before sharing it. Leadership at METR and Redwood Research received a 38-page version. The public received only a short summary. METR said it lacked the right to disclose the redacted sections on its own. The group stated it wants to push for greater public transparency in future audits.