Anthropic Wants Courts to Block Risky AI Models

Anthropic has suggested in a letter to California Governor Gavin Newsom that the government should be able to ask courts for injunctions to block AI models that pose serious risks to the public.

In September, Newsom formed an AI expert panel to study safety rules, and Anthropic sent in these ideas. The company warned that an emergency kill switch for a whole model could give officials a false sense of security. It prefers targeted steps, like shutting down a broken AI agent or pausing access for a specific customer.

Earlier this year, the US government asked Anthropic to limit foreign access to two of its cybersecurity models, but the Commerce Department later pulled back that order. US lawmakers have also discussed bills to let the government block risky AI models, and OpenAI shared its own ideas with Newsom in October.

Anthropic is now preparing to go public in the US. If it succeeds, it could become one of the most influential IPOs in recent years.