Large Language Model Influence on Diagnostic Reasoning: A Randomized Clinical Trial
In a small randomized trial of 50 physicians in JAMA Network Open, ChatGPT Plus alone, given the case studies, reached an accuracy of more than 92%. Doctors using ChatGPT were accurate about 76% of the time, against about 74% for doctors without it. A retrospective study of 120 Intensive Care Unit admissions found diagnostic accuracy of 88.3% for physicians and 85.0% for ChatGPT-4, with no significant difference. A study of probabilistic reasoning found that when test results were positive, the chatbot's accuracy was comparable to clinicians in most cases.