The use of AI in healthcare is designed to improve care delivery and augment the decisions of providers to enhance patient outcomes. When deployed in clinical settings, the interaction between providers and AI is a critical component for measuring and understanding the effectiveness of these digital tools on broader health outcomes. Even in cases where AI algorithms have high diagnostic accuracy, healthcare providers often still rely on their experience and sometimes gut feeling to make a final decision. Other times, providers rely unquestioningly on the outputs of the AI models, which leads to a concern about over-reliance on the technology. The purpose of this research was to understand how reliant drug shop dispensers were on AI-powered technologies when determining a differential diagnosis for a presented clinical case vignette. We explored how the drug dispensers responded to technology that is framed as always correct in an attempt to measure whether they begin to rely on it without any critical thought of their own. We found that dispensers relied on the decision made by the AI 25 percent of the time, even when the AI provided no explanation for its decision.

翻译：人工智能在医疗保健中的使用旨在改善医疗服务交付，并增强医疗提供者的决策能力，以提升患者预后。当这些技术部署在临床环境中时，提供者与AI之间的互动是衡量和理解这些数字工具对更广泛健康结果有效性的关键组成部分。即使AI算法具有高诊断准确性，医疗提供者往往仍依赖自身经验（有时甚至是直觉）做出最终决策。而在其他情况下，提供者会不加质疑地依赖AI模型的输出，从而引发对技术过度依赖的担忧。本研究旨在探究药品配发人员在面对临床案例摘要进行鉴别诊断时，对AI驱动技术的依赖程度。我们考察了药品配发人员如何回应被设定为“始终正确”的技术，以衡量他们是否开始在没有自身批判性思考的情况下依赖该技术。研究发现，即使AI未对其决策提供任何解释，配发人员在25%的情况下仍会依赖AI所做的决策。