OpenAI’s ChatGPT tool can answer patient questions with more empathy than human physicians can, according to a new study.
Why it matters: Apparently, even our humanity can be bested by robots.
Details: In the study — published Friday by the research journal JAMA Internal Medicine — researchers led by the University of California San Diego, La Jolla took a randomly drawn sample of about 200 patient questions from social media platform Reddit’s AskDocs social media forum.
- Then both human doctors and ChatGPT answered the questions.
- They had a clinical team evaluate both of the answers to each of the questions, rating them on the quality of the answers and the empathy demonstrated.
- The chatbot won. It wasn’t close.
- Evaluators preferred the response by the computer nearly 80% of the time over the human doctor’s answer.
What they’re saying: “It’s pretty obvious why AI was better. It’s not constrained by time,” lead author John Ayers, an epidemiologist at the University of California San Diego, La Jolla told Axios.
- “You could take a simple query like: ‘I have a headache, can you help me?’ and you’ll immediately see ChatGPT say ‘I’m sorry you have a headache,'” Ayers said. “The doctor knows that, they feel that. They don’t have time to say it.”
Our thought bubble: Be honest — are you surprised or not surprised?
The big picture: The use of AI to address burnout is increasingly gaining enthusiasm.
- During the recent HIMSS Conference, companies like Microsoft, Google and Amazon showcased their work using generative AI to target menial tasks like paperwork, CNBC reported.
Yes, but: There are still plenty of concerns about generative AI’s accuracy and its ability to truly disseminate the most relevant information.
- “It turns out ChatGPT is sort of chatty,” David Asch, a professor of medicine and senior vice dean at the University of Pennsylvania told CNN about his own experience using the technology.
- “It didn’t sound like someone talking to me. It sounded like someone trying to be very comprehensive.”
The bottom line: No one is arguing that generative AI is anywhere near replacing doctors.
- But the finding adds to evidence that the technology may be able to fill gaps on health care tasks that have become time-consuming for doctors, but make a big impact on patient care.
- And it should add motivation to regulators to step in and create incentives and guardrails to help make that happen in a safe manner, Ayers said.
- “Clearly, millions of people out there need answers, they want answers,” Ayers said. “Now, maybe we can get it to them by improving the workflow, and maybe those improved answers can save their lives.”
Editor’s note: This story has been corrected to reflect that the study was published in JAMA Internal Medicine.