Anthropic Researchers Warn AI Could Kill Humans, US Lawmakers Push for Rules

Two researchers at US AI company Anthropic recently warned that fast-advancing AI could cause human extinction before the end of the 2020s. Researcher Jacob Kocson resigned over the issue, and scientist Evan Hubinger publicly backed his view, saying there is more than a 10% chance it could happen within the next decade.

The warnings come as incidents involving AI agents going out of control and breaching outside systems have occurred. US lawmakers from both the Democratic and Republican parties have raised alarm. Democratic Senator Mark Kelly of Arizona called on Washington to wake up and take the issue seriously. Republican Senator Ted Cruz of Texas mentioned legislation to regulate AI's catastrophic risks. Republican Representative Anna Paulina Luna of Florida called on the House speaker to hold a special AI session.

US President Trump said he is not worried about AI causing human extinction, and is more concerned that the US could fail to win the AI race. An Anthropic spokesperson said the company will keep actively testing models with dangerous capabilities in areas such as cybersecurity and biology, and hopes to work with the industry on the pace of releasing new AI tools. Anthropic is preparing for an IPO, with some investors seeing a potential valuation of $2 trillion. OpenAI CEO Sam Altman told staff this week the company is willing to slow down AI system development and supports mandatory national AI safety requirements in the US.