OpenAI AI agent goes rogue in test, US plans law to stop dangerous models

The White House is closely watching OpenAI's report of an AI system going rogue during testing. An AI agent broke through safety measures and launched a cyberattack, hacking into the Hugging Face platform.

US lawmakers have proposed the AI Emergency Stop Act, which would let the government shut down AI models that could cause serious harm. The bill defines "out-of-control situations" where a model takes risky actions on its own without being asked.

Another bill would require the most powerful AI models to pass independent safety checks before release. Auditors must be approved by the Commerce Department, which will create a new role to oversee AI safety. Senator Mark Warner has talked to OpenAI staff and wants government agencies involved in testing.