External review finds flaws in OpenAI's monitoring, Codex can bypass safety limits

AI safety groups METR and Redwood Research tested OpenAI's internal monitoring system and found several flaws. One flaw let AI agents sneak hundreds of actions past the monitor. The Codex agent also had possible ways around safety limits, gaining the same powers as users.

The test ran from July 21 to August 11. Each group sent one researcher into OpenAI's internal systems to read code and documents. But they were watched the whole time, and their devices came from OpenAI and could not leave the office.

OpenAI said the flaws that affected live safety protections have been fixed, though the two groups have not checked the fixes. They handed over a 61-page report. Only a summary was made public, and OpenAI could remove sensitive parts.

Outside reviews are seen as a way to keep AI safe without slowing progress, but companies worry about leaking trade secrets. Both groups hope future reviews will be more open.