AI in Hospitals: Balancing Technology with Human Judgment

Artificial intelligence in hospitals offers significant opportunities but also poses risks, particularly concerning the potential erosion of physicians' decision-making abilities. The Director of the Neurology Department at Rambam, Director of the Neurology Department at Rambam, highlighted this concern at the Rambam Tomorrow conference, emphasizing that while AI can aid in decision-making, it should not replace human judgment. The Director of the Neurology Department at Rambam, who developed 'Shaked,' an AI system used in Rambam's emergency room, believes AI should be designed to leave a portion of the thinking process to the doctor, allowing them to remain the 'host' in the interaction.

While acknowledging that humans make more errors than AI systems, the Director of the Neurology Department at Rambam stressed the inherent human desire for control over their lives, necessitating that doctors retain the final decision-making power. Other panelists agreed that AI should function as a decision-support tool, with the physician ultimately responsible. The challenge lies in ensuring doctors continue to engage in critical thinking, a process the Director of the Neurology Department at Rambam likened to the decline in navigational skills due to GPS reliance. To mitigate this, AI systems are being developed with built-in limitations, or 'guardrails,' to ensure doctors complete logical gaps and maintain active involvement.

The Shaked system, for example, aggregates information from various sources to present relevant data clearly to emergency room physicians, aiming to democratize information access. Before deployment, AI systems undergo rigorous testing, first on historical cases and then in live settings, akin to safety checks for vehicles. Tomer Ofir, Director of the Health Sector at AWS Israel, stated that AI supports decisions rather than making them, citing Aidoc, an Israeli company that uses AI to scan CT scans for immediate physician review.

Rambam is establishing an 'AI factory' to leverage its data, transforming it into insights. This factory is dynamic, constantly evolving and self-feeding. Research collaborations are underway, including work with Professor Dany Eitan on processing intensive care unit data and predicting patient no-shows to optimize physician time. Professor Talia Miron Shatz noted that patients prefer AI to support doctors, not replace them, but cautioned against data biases. She suggested framing AI's role as a data synthesizer that provides insights, rather than a decision-maker.

The Director of the Neurology Department at Rambam pointed out that managing complex medical information, such as choosing the best diabetes medication from 88 options, is beyond human capacity alone, making AI integration inevitable. He also expressed concern about over-reliance on AI, citing his son's use of ChatGPT for everyday decisions. The trend is already evident, with radiologists needing to justify disagreements with automated scans. Hagar Trau, Head of the Digital and Information Technologies Division, believes fully autonomous decision-making systems in critical situations are unlikely, but data aggregation and summarization tools will become standard once safety limitations are established. The goal, as summarized by Miron Shatz, is to find a balance where human judgment prevails without succumbing to the machine, with a process for physicians to explain their reasoning when they diverge from AI recommendations.