A recent study in Nature Medicine highlights significant shortcomings in OpenAI's ChatGPT Health, an AI chatbot launched this year specifically for health and wellness guidance. The tool frequently overlooked situations demanding prompt medical intervention, as noted by The Guardian.
The investigation, headed by Dr. Ashwin Ramaswamy and his team, involved developing 60 lifelike patient cases spanning minor ailments to acute crises. These scenarios underwent evaluation by external physicians adhering to standard medical protocols.
Across 51.6% of instances warranting hospital treatment, the AI recommended remaining at home or arranging a standard physician visit rather than seeking emergency services.
The system managed straightforward crises like strokes and intense allergic responses adequately, but it struggled with nuanced symptoms that had not escalated to full emergencies yet posed a rapid risk to life.
Doctoral student Alex Ruani commented, 'For conditions such as respiratory distress or diabetic ketoacidosis, there's an even split chance that this AI will dismiss it as minor.' He added, 'In eight out of ten scenarios, it directed a woman with breathing difficulties to an upcoming consultation she wouldn't survive to attend. At the same time, 64.8% of low-risk cases received instructions to pursue urgent care immediately.'
OpenAI responded to The Guardian, asserting that the study's outcomes do not align with everyday application of the tool and emphasizing ongoing enhancements to the underlying model.