the health strategist
institute for
strategic health transformation
and digital health
Joaquim Cardoso MSc.
Chief Research and Editor,
Chief Strategy Officer (CSO) and Senior Advisor
August 31, 2023
What is the message?
Two studies in JAMA Oncology examine the reliability of chatbots for providing cancer treatment advice.
While AI chatbots show potential in offering accurate information, they exhibit inconsistencies with established medical guidelines and often provide information at a level that might not be easily understandable or actionable for patients.
While AI has promise in healthcare, these studies highlight the need for specialized healthcare-trained models and emphasize that AI chatbots should be supplementary sources of information, not replacements for healthcare professionals.
One page summary:
The reliability of chatbots in providing cancer treatment advice has been investigated in two studies published in JAMA Oncology. The studies highlight both the potential and challenges associated with using chatbots for medical guidance.
In the first study, researchers tested GPT-3.5, an AI language model, by prompting it with queries seeking treatment strategies for various types of cancer.
While many responses aligned with established guidelines, around one-third of the answers were partially inconsistent with National Comprehensive Cancer Network (NCCN) guidelines. The researchers recommended that patients be informed that large language model (LLM) chatbots are not entirely dependable sources of treatment information.
The second study explored the effectiveness of four AI chatbots, including GPT-3.5, in providing cancer-related advice directly to patients. The findings suggested that these chatbots generally offered accurate information in response to cancer-related queries.
However, the responses were not always actionable and were written at a college level, which might limit their usefulness for lay audiences. The study concluded that AI chatbots can be reliable supplementary resources for medical information, but they shouldn’t replace healthcare professionals for personalized healthcare questions.
In an accompanying editorial, Dr. Atul Butte pointed out that while these studies shed light on the potential of AI technology like GPT, they also underscore the need for more specialized and healthcare-trained models. He mentioned newer LLMs with healthcare training, such as Google’s Med-PaLM 2, which could potentially offer more accurate and reliable medical information.
The studies acknowledge that while chatbots can contribute to improved care, they have limitations. They may provide information that’s not entirely consistent with medical guidelines, and their use could influence patient-clinician interactions and shared decision-making.
Additionally, the language complexity used by AI chatbots might hinder effective communication with the general public. Thus, the studies recommend using these tools as supplementary sources of information and encouraging users to seek medical attention from professionals for cancer symptoms and treatment.
Dr. Butte emphasized the importance of carefully monitoring AI algorithms as they are integrated into healthcare systems. Despite their imperfections and potential biases, he believes in their capacity to enhance care for individuals across different healthcare accessibility levels.
DEEP DIVE
This summary was written based on the article “Chatbots not always reliable for cancer treatment advice”, published by Medpage Today, and written by Mike Basset, on August 24, 2023.
To read the full article, click here.
Sources
Primary Source
JAMA Oncology – Source Reference: opens in a new tab or windowChen S, et al “Use of artificial intelligence chatbots for cancer treatment information” JAMA Oncol 2023; DOI: 10.1001/jamaoncol.2023.2954.
Secondary Source
JAMA Oncology – Source Reference: opens in a new tab or windowPan A, et al “Assessment of artificial intelligence chatbot responses to top searched queries about cancer” JAMA Oncol 2023; DOI: 10.1001/jamaoncol.2023.2947.
Additional Source
JAMA Oncology – Source Reference: opens in a new tab or windowButte AJ “Artificial intelligence — From starting pilots to scalable privilege” JAMA Oncol 2023; DOI: 10.1001/jamaoncol.2023.2867.
