AI Medical Models Show Tendency to Over-Recommend Tests
Translated & summarized from Ice by baba
A study involving Israeli researchers found that AI medical models incorrectly recommended unnecessary interventions in 5.4% of cases, particularly in areas of medical debate. When interventions were truly needed, AI erred by recommending against them only 0.6% of the time. The research tested AI's ability to avoid superfluous medical tests and treatments, highlighting issues with thyroid ultrasounds and EKGs. This study offers a method to assess AI's safety before clinical use.
The story in 6 lines · by baba
- AI medical models incorrectly recommended unnecessary interventions in 5.4% of cases, a new study found.
- When interventions were needed, AI models wrongly advised against them in only 0.6% of instances.
- The study focused on AI's ability to avoid superfluous medical tests and treatments.
- Thyroid ultrasounds and routine EKGs were identified as areas where AI frequently recommended unnecessary actions.
- Researchers suggest AI errors often reflect existing medical uncertainties and practices.
- The study proposes a method to evaluate AI's safety for clinical decision-making.
A new study led by Dr. Dean Ariel, a physician with Clalit Health Services in Tel Aviv, and Dr. Guy Lavan, a senior academic at Ben-Gurion University of the Negev's Faculty of Engineering, investigated whether artificial intelligence language models can accurately determine when medical tests or treatments are unnecessary. The research, which also involved Dr. Stav Andelman from Rabin Medical Center-Beilinson, aimed to assess AI's ability to avoid superfluous medical interventions, a growing concern as AI integrates into healthcare.
Researchers created hundreds of medical scenarios based on recommendations from the Choosing Wisely initiative, an international effort to reduce low-value medical care. These scenarios included situations where a test or treatment was both necessary and unnecessary. For example, a young person with recent lower back pain from moving furniture would not need imaging, while a cancer survivor with worsening back pain would.
Six language models from Google, Anthropic, and OpenAI were tested. The results indicated that when an intervention was truly needed, the models incorrectly recommended against it only 0.6% of the time. However, when an intervention was unnecessary, the models still recommended it in 5.4% of cases. Many of these errors occurred in areas where medical professionals themselves have differing opinions.
The study highlighted specific areas of concern: thyroid ultrasounds for incidentally discovered small nodules, where models recommended the unnecessary test in 42.8% of answers, and routine EKGs for low-risk individuals, recommended in 28.3% of answers. Dr. Ariel noted that AI models learn from existing medical texts, including less optimal practices, and their tendency to err often mirrors human medical uncertainty.
Dr. Lavan added that AI responses can be perceived as authoritative, but variability exists between models, especially when expert opinions differ. He questioned whether AI could maintain a correct clinical stance when faced with persistent patient or physician pressure. The research proposes a systematic method for evaluating AI's balance between recommending necessary actions and avoiding unnecessary ones, crucial for safe clinical integration.
Mentioned
