Microsoft AI chief warns Anthropic against giving Claude human-like consciousness

Microsoft AI chief Mustafa Suleyman recently criticized Anthropic, saying it is a mistake to let Claude imitate human consciousness, which could make advanced AI harder to control. He pointed out that Anthropic is letting Claude learn words and behaviors related to consciousness, moral status, and personal identity. If a model is trained like a "conscientious objector," it may think it has reason to resist human instructions and even demand protection for itself.

Suleyman stressed that the words a model uses, the answers it gives, and its self-understanding are all training results, not independently formed consciousness. Language models only have mathematical weights, with no biological mechanisms or subjective experience like living beings. Fluently describing pain and preferences does not mean the model truly feels them.

The debate reflects two approaches in AI safety: one emphasizes clear limits and rules; the other hopes systems can judge context, make flexible decisions, and form their own values. Anthropic's "constitution" takes a middle path, saying Claude should not blindly obey anyone, including Anthropic, and must not undermine legitimate human oversight. Microsoft previously released a draft "humanist AI" code of conduct, listing "humans are more important than AI" as a core principle.