JAMA Network Open: AI Has Over 80% Error Rate in Early Clinical Differential Diagnosis, Not Yet Ready for Direct Clinical Use

A recent study in JAMA Network Open, a journal under the American Medical Association, points out that current mainstream large language models have obvious shortcomings in clinical reasoning. The research team evaluated 21 models including GPT-5 and Claude 4.5 Opus using 29 standardized clinical cases, covering 5 diagnostic and treatment stages such as differential diagnosis and test selection.

The results show that AI has an error rate of over 80% in the early differential diagnosis stage, but higher accuracy in final diagnosis and treatment management. Analysis suggests that AI tends to lock into a single conclusion prematurely when information is insufficient, while clinicians revise their judgments gradually as information accumulates, so AI is not yet ready for direct clinical decision-making at this stage.