Study says AI chatbots could safely help diagnose patients before GPs
Though small the study found that AI’s suggested diagnoses matched the doctors in more than half of cases.
A new study found AI chatbots could help diagnose patients days before they see a GP.
The study was small though and while more research is needed, it appears the technology could help family doctors prepare for patient visits, and may even change their approach to treatment in some cases.
Researchers in the US looked at online conversations from 98 patients at a GP clinic in Boston, who talked to an AI chatbot up to five days before their scheduled appointment.
The chatbot gave each patient a list of potential diagnoses and, in some cases, next steps to discuss at their upcoming appointments.
Doctors monitored the conversations in real time and were also sent the chat transcript before the appointment.
The study, published in The Lancet, found that AI’s suggested diagnoses matched the eventual confirmed diagnosis in more than half of cases.
Safety issues
The correct diagnosis also appeared when the chatbots’ top three suggestions were taken into account in 75% of cases.
None of the conversations triggered safety stops.
Patient attitudes towards AI in primary care also improved after interacting with the chatbot, researchers said.
However, one conversation was later rated “somewhat harmful” by a doctor after the chatbot listed a possible cancer diagnosis which caused the patient anxiety.
In another conversation, the chatbot hallucinated, and clinical information was added in five interactions.
Researchers said: “Although further research is needed, this study shows the initial feasibility of conversational AI in a real-world setting – assessed via conversation safety and quality, as well as user acceptance- and represents a crucial step towards clinical translation.”
Elsewhere, doctors reviewed the AI transcript beforehand and filled out a questionnaire for 44 cases in the study.
In 75% of cases, medics said the AI chatbot helped them prepare for patient visits.
They also said it may have changed their approach for more than half of cases.
Researchers said the findings suggests that patient-facing chatbots “can engage in safe and meaningful clinical conversations in real workflows.
However they stress the evidence “does not yet support autonomous use for diagnosis, triage, or management”.
They called for future studies to examine the safety of diagnoses, communication between patients and clinicians, doctors’ workloads, cost, and the level of oversight required for safe use.
Professor Victoria Tzortziou Brown, president of the Royal College of GPs, said: “Research into how AI performs with real patients in real clinical settings is welcome, particularly as much of the evidence so far has come from simulated cases.
“This study provides some interesting early findings, but it is small, was conducted in a single US practice, and importantly every chatbot conversation was closely supervised by a doctor, with patients also speaking to that doctor afterwards. It therefore cannot tell us how safe or effective this approach would be without that level of clinical oversight.”
Prof Brown said the relationships between GP and patients over time also play an important role in diagnosis.
“GPs bring clinical judgment, an understanding of a patient’s history and circumstances and, through continuity of care, knowledge built up over time,” she added.
“AI should support that relationship and clinical decision-making, not replace them.
“Before tools like this are introduced into NHS care, we need larger, independent studies comparing them with current care and looking at safety, inequalities, impact on GP workload and, crucially, whether they actually improve outcomes for patients. Alongside that evidence, we need appropriate regulation and clear clinical accountability.”