A recent study by researchers from Harvard Medical School and Stanford University revealed that OpenAI's ChatGPT o1 (in beta "Preview-01") AI model may outperform doctors in diagnosing complex medical conditions.
During the study, the model underwent a comprehensive series of medical diagnostic tests, and the results showed that it achieved qualitative leaps compared to previous versions. The "o1-preview" model was able to accurately diagnose 78.3% of the cases analyzed.
In a direct comparison across 70 specific cases, the system's accuracy rose to 88.6%, significantly outperforming the previous GPT-4 system, which scored 72.9%. The system also demonstrated its high proficiency in medical reasoning, scoring high in 78 out of 80 cases according to the R-IDEA scale used to assess the quality of reasoning.
The researchers noted that the model's training data may have included some of the cases used in the study, but the model's performance remained high when tested on new cases it had not previously encountered, with a slight decline in performance.
The researchers explained that the model's detailed answers contributed to its improved rating, stressing that the study was limited to its individual performance without examining how it collaborated with physicians.
The Previo-01 model excels at critical thinking tasks, such as making diagnoses and making treatment recommendations, but struggles with abstract tasks, such as estimating probabilities.
OpenAI recently announced the release of the full, final version, O1, as well as the new version, O3, which showed significant improvements in analytical reasoning.
Read also : Tips for breastfeeding mothers to keep them hydrated
0 Comments