Ex-UK data scientist tests AI in Civilization VI, reveals blind spots in sensing and execution

Liam Wilkinson, a former data scientist at the UK Prime Minister's office, built a system over a weekend to plug AI models like Claude, GPT, and Gemini into Civilization VI. Over 23 games, the AIs saw only text and coordinates, not visuals.

The tests exposed major flaws. In one game, Claude played Portugal and feared France's cultural victory. It spent 50 turns building a nuke to destroy Toulouse, but missed France's diplomatic win. The AIs checked the game state only 1-2% of the time and executed plans just 48-66% of the time.

Wilkinson says the real problem isn't smarts—the models scored 99.26% on policy tests—but sensing and doing. AIs can't see what they don't check and can't do what they plan. This points to an engineering challenge: making AIs truly perceive and act, not just think bigger.