New research has raised serious concerns about the safety of using artificial intelligence (AI) chatbots to seek medical advice. The study highlights the dangers of depending on AI systems when making health-related decisions, citing their frequent tendency to provide inaccurate and inconsistent information.
Conducted by experts from the Oxford Internet Institute and the Nuffield Department of Primary Care Health Sciences at the University of Oxford, and published in Nature Medicine, the study underlines that current AI technology is not yet suitable to replace human physicians.
Dr. Rebecca Payne, a co-author and practicing GP, emphasized, “Despite the hype, AI is not ready to assume the role of a doctor.” She cautioned patients that consulting large language models about symptoms can be risky, sometimes resulting in incorrect diagnoses or failure to identify urgent medical needs.
READ MORE: My uncle was kidnapped by Nazis - now he’s left us £100k from new life in West Midlands
READ MORE: ‘Warm and Colourful’ £42 Next Cherry Print Active Quarter Zip Borg Jacket is Ideal for Spring Dog Walks
The study involved nearly 1,300 participants who were asked to identify possible health conditions and recommend treatments based on various scenarios. Some participants used AI-powered large language models (LLMs) to obtain potential diagnoses and advice, while others relied on conventional methods such as in-person GP consultations.
Analysis revealed that AI chatbots frequently offered a confusing blend of accurate and misleading information, making it difficult for users to discern the reliability of the advice given. Although these AI systems perform well on standardized medical knowledge tests, applying them in real-world medical situations poses significant risks to users seeking help for their own symptoms.
Dr. Payne noted, “These findings highlight the challenge in creating AI systems capable of reliably supporting people in sensitive, high-stakes fields such as healthcare.”
Andrew Bean, lead author from the Oxford Internet Institute, added that the research demonstrated the difficulty AI faces in effectively interacting with humans. He expressed hope that the findings would guide the development of safer and more beneficial AI tools in the future.