Oxford Internet Institute study finds 'AI chatbots less helpful than search engines for medical advice'

An Oxford-led study showing that current AI chatbots are not reliably safe or useful for lay people making medical decisions and can mix solid guidance with misleading advice in ways users struggle to interpret.

What the study did

Researchers from the Oxford Internet Institute and Nuffield Department of Primary Care Health Sciences ran one of the largest user studies to date on AI medical advice, with nearly 1,300 UK participants facing common symptom scenarios (for example, severe headaches, post‑natal fatigue).

Participants had to decide what to do (self‑care, see a GP, go to A&E), with some using large language model (LLM) chatbots and others relying on traditional sources such as web search or their own judgement.

Main findings

Using an LLM did not lead to better health decisions than traditional methods; participants with chatbot access were, on average, no more accurate about what was wrong or what level of care to seek.

Chatbot outputs often combined accurate and inaccurate information, so users found it hard to tell which parts to trust, particularly when the model listed multiple possible conditions.

Many users did not know which details of their symptoms to share or how to phrase questions, and small changes in wording produced inconsistent, sometimes contradictory answers.

Why this matters

The study highlights a gap between LLMs’ strong performance on standardised medical knowledge tests and their weaker performance in messy, real‑world interactions with non‑experts.

Researchers argue that, despite excitement about AI in healthcare, general‑purpose chatbots are not ready to act as a doctor or a trusted source of medical decision support for patients.

Implications and recommendations

The authors call for cautious deployment of AI chatbots in health, stressing that current systems can pose risks by giving inconsistent or unsafe advice that may delay appropriate care.

They suggest that future work should focus on designing safer, more tightly constrained medical AI tools and better interfaces that guide people on what information to provide and how to interpret responses.

About the Study

A new study from the Oxford Internet Institute and the Nuffield Department of Primary Care Health Sciences at the University of Oxford, carried out in partnership with MLCommons and other institutions, reveals a major gap between the promise of large language models (LLMs) and their usefulness for people seeking medical advice. While these models now excel at standardised tests of medical knowledge, they pose risks to real users seeking help with their own medical symptoms. 

About the Oxford Internet Institute (OII)   

The Oxford Internet Institute (OII) has been at the forefront of exploring the human impact of emerging technologies for 25 years. As a multidisciplinary research and teaching department, we bring together scholars and students from diverse fields to examine the opportunities and challenges posed by transformative innovations such as artificial intelligence, large language models, machine learning, digital platforms, and autonomous agents. 

About the University of Oxford  

Oxford University has been placed number one in the Times Higher Education World University Rankings for the tenth-year running, and number two in the QS World Rankings 2022. At the heart of this success are the twin-pillars of our ground-breaking research and innovation and our distinctive educational offer. Oxford is world-famous for research and teaching excellence and home to some of the most talented people from across the globe.    

Source: https://www.oii.ox.ac.uk/news-events/new-study-warns-of-risks-in-ai-chatbots-giving-medical-advice/