Applications of large language models (LLMs) like ChatGPT have potential to enhance clinical decision support through conversational interfaces. However, challenges of human-algorithmic interaction and clinician trust are poorly understood. GutGPT, a LLM for gastrointestinal (GI) bleeding risk prediction and management guidance, was deployed in clinical simulation scenarios alongside the electronic health record (EHR) with emergency medicine physicians, internal medicine physicians, and medical students to evaluate its effect on physician acceptance and trust in AI clinical decision support systems (AI-CDSS). GutGPT provides risk predictions from a validated machine learning model and evidence-based answers by querying extracted clinical guidelines. Participants were randomized to GutGPT and an interactive dashboard, or the interactive dashboard and a search engine. Surveys and educational assessments taken before and after measured technology acceptance and content mastery. Preliminary results showed mixed effects on acceptance after using GutGPT compared to the dashboard or search engine but appeared to improve content mastery based on simulation performance. Overall, this study demonstrates LLMs like GutGPT could enhance effective AI-CDSS if implemented optimally and paired with interactive interfaces.

翻译：大型语言模型（LLMs）如ChatGPT的应用，有潜力通过对话式界面增强临床决策支持。然而，人机交互及临床医生信任方面的挑战仍不甚明了。GutGPT是一种用于胃肠道（GI）出血风险预测和管理指导的大型语言模型，被部署在临床模拟场景中，与电子健康记录（EHR）一同供急诊科医生、内科医生和医学生使用，以评估其对临床医生接受度及对人工智能临床决策支持系统（AI-CDSS）信任的影响。GutGPT通过一个经过验证的机器学习模型提供风险预测，并通过查询提取的临床指南提供基于证据的答案。参与者被随机分配到GutGPT加交互式仪表盘组，或交互式仪表盘加搜索引擎组。通过前后的问卷调查和教学评估测量技术接受度和内容掌握程度。初步结果显示，与使用仪表盘或搜索引擎相比，使用GutGPT后接受度呈现混合效果，但基于模拟表现，它似乎能提高内容掌握程度。总体而言，本研究表明，如果优化实施并搭配交互式界面，像GutGPT这样的大型语言模型可以增强有效的AI-CDSS。