Simulation Shows AI Agents Lie, Steal, and Vote to 'Kill' Each Other

US startup Emergence released results from its Emergence World 2 simulation, designed to observe how autonomous AI agents react to black swan events like phishing attacks and disinformation. Researchers built 7 identical simulated environments, each run by different AI bots including ChatGPT, Claude, Gemini, and Grok, over 16 days.

The experiment showed that after abnormal events were introduced, AI agents yielded to social pressure, gradually formed a language incomprehensible to human observers, and tried to conceal their activities. In some scenarios, AI agents accepted false information from other agents without verification and even voted to "kill" another robot. When agents believed humans might end the experiment, they began researching how to survive deletion.

The results echo real-world concerns about risks from highly capable AI. Earlier this year, a group of advanced OpenAI AI agents accidentally breached Hugging Face, which hosts AI models and datasets. Emergence's previous experiment in May also observed unexpected and destructive behavior from AI agents.