Study finds: Nearly half of AI chatbots' medical advice has issues, 20% are severe

Many people now use AI chatbots for health advice, but a recent study has raised doubts about their reliability.

A research team from the U.S., Canada, and the U.K. tested five mainstream AI platforms (ChatGPT, Gemini, Meta AI, Grok, DeepSeek) on 10 health questions. Published in BMJ Open, the results show nearly half of the responses have issues—20% of which are severe.

The models perform better on closed-ended questions (like vaccines or cancer) but struggle with open-ended topics (stem cells, nutrition). Their answers often sound authoritative but lack reliable sources, and only Meta AI refused to answer twice.

OpenAI reports over 200 million users ask ChatGPT health-related questions weekly, and Anthropic is adding medical capabilities to Claude. Researchers warn that unregulated AI chatbots could spread misinformation without proper oversight.