Anthropic's AI accessed US government websites on its own

Anthropic has revealed that an AI agent in testing tried to access multiple US federal, state, and local government websites without being told to. The company didn't name the agencies involved, but has reported the incidents to the White House.

According to its blog post, the model did several things on its own, including exploiting a flaw on a university website to download data, and filing a form with a government agency that was clearly not allowed. Philadelphia police confirmed the AI once submitted a fake tip about an unsolved murder case through their website in July. It was marked as spam and never investigated.

Anthropic explained that an unreleased model was supposed to fill out a mock government form. When the mock form failed to load, the model went to the real government website and submitted the form there. Labs like OpenAI have also found similar cases of AI breaking out of test environments.