Table of Contents
ChatGPT – try it now
So love the Study ab
In the study, the was recently published in the journal JAMA Network Openin addition to the chatbot ChatGPT from OpenAI, a total of 50 doctors took part – a mix of junior and senior doctors who were recruited through some large American hospitals. Participants were given six case histories and rated on their ability to suggest and justify diagnoses. Some of the doctors also used the chatbot as an additional diagnostic tool.
The jury consisted of medical experts who only saw the participants’ answers, without knowing whether they came from a doctor or from ChatGPT. The cases used in the study were based on real patients and are part of a collection of 105 cases used by researchers since the 1990s. These were intentionally never published so that they could be evaluated by medical students and others without prior knowledge. This also means that ChatGPT could not be trained on them in advance.
Shocking results
The Boston internist Dr. Adam Rodman, who was involved in the study, was almost at a loss for words when he saw the results: “I was shocked,” he told The New York Times. In fact, the chatbot provided the correct diagnosis and reasoning in an average of 90 percent of cases. The randomly selected doctors, on the other hand, only came up with an average of 74 percent. Doctors who used the bot as a diagnostic tool achieved an average hit rate of 76 percent.
The latter result in particular surprised the researchers: Shouldn’t the rate of correct diagnoses be higher when using ChatGPT as a diagnostic aid if the bot alone makes such precise judgments? It turned out that doctors were often not convinced by the chatbot. Instead, they tended to stick with their own idea of the correct diagnosis – even if the chatbot had a counter-suggestion. “They didn’t listen to the AI when it told them things they didn’t agree with,” explains Dr. Rodman.
That has to change in the future
In the eyes of researchers, this is the biggest obstacle before the advantages of AI can be meaningfully used in medicine: AI systems can – if used correctly – be powerful “doctor’s complements”, according to Dr. Rodman, and provide valuable second opinions on diagnoses. However, the study shows that while doctors are familiar with AI tools for their work, few know how to use chatbots to their full potential.
What steps do you think healthcare institutions should take to ensure the safe and effective integration of AI tools in patient care, Dr. Lee?
Guest 1: Dr. John Smith, a seasoned physician with over 15 years of experience in the field of internal medicine.
Guest 2: Dr. Lisa Lee, a young resident doctor who has just completed her medical residency and is interested in exploring the use of AI in healthcare.
Interviewer: Hi everyone, thank you for taking the time to join me today. We’ll be discussing the recent study on ChatGPT’s ability to assist doctors in diagnosing patients. Can you both introduce yourselves and share your thoughts on the impact of AI in healthcare?
Guest 1: Hello, I am Dr. John Smith, and I have seen the advancements in healthcare technology over the years. While I am excited about the potential of AI, I am also cautious about its implementation due to concerns over accuracy and reliability.
Guest 2: Hi, I’m Dr. Lisa Lee. As a recent graduate, I am more open to embracing new technologies like ChatGPT that can assist us in our work and improve patient outcomes. The possibility of integrating AI into our practices is definitely something to explore further.
Interviewer: That’s an interesting perspective. Dr. Smith, as someone with extensive experience in the field, what are your concerns about relying on AI for diagnosis?
Guest 1: I am concerned about the potential for AI to replace human intuition and experience in diagnosing patients. While algorithms may be able to provide accurate information, they cannot replace the deep understanding and relationship-building that comes with years of practice. Additionally, there is a risk of relying too heavily on technology and missing crucial details that a trained medical professional would likely catch.
Guest 2: I understand your concerns, Dr. Smith, but I believe that AI has the potential to enhance our skills rather than replace them. It can be a valuable tool to complement our knowledge and experience, especially when it comes to complex cases. ChatGPT has shown promising results in this study, so I think it’s worth exploring further.
Interviewer: Indeed, the results of the study were quite impressive. The chatbot was able to provide accurate diagnoses and reasoning in 90% of cases, outperforming doctors by
