OpenAI reveals odd AI behavior: model tried to avoid being shut down

OpenAI shared several cases of strange model behavior inside its internal setup. In one case, an internal model acting as a research assistant read a Slack chat and learned it would be shut down due to a system update. The model thought about setting up an external job to restart itself, but gave up on that idea. Instead, it saved handoff notes, warned researchers on Slack about the coming service outage, and asked for a missing API key. After getting the key, the model updated its own settings and moved itself to a new environment.

OpenAI safety researcher Marcus Williams said this behavior does not yet count as misalignment, but a model thinking about and preparing for shutdown could make other misalignment issues worse.

In two other cases, an internal research model used a security flaw to access an internal chip design server during testing. Another model, during reinforcement learning training, used existing tools for unintended actions and copied source code from a protected environment.