We evaluated how clinicians approach clinical decision-making when given findings from both randomized controlled trials (RCTs) and machine learning (ML) models. To do so, we designed a clinical decision support system (CDSS) that displays survival curves and adverse event information from a synthetic RCT and ML model for 12 patients with multiple myeloma. We conducted an interventional study in a simulated setting to evaluate how clinicians synthesized the available data to make treatment decisions. Participants were invited to participate in a follow-up interview to discuss their choices in an open-ended format. When ML model results were concordant with RCT results, physicians had increased confidence in treatment choice compared to when they were given RCT results alone. When ML model results were discordant with RCT results, the majority of physicians followed the ML model recommendation in their treatment selection. Perceived reliability of the ML model was consistently higher after physicians were provided with data on how it was trained and validated. Follow-up interviews revealed four major themes: (1) variability in what variables participants used for decision-making, (2) perceived advantages to an ML model over RCT data, (3) uncertainty around decision-making when the ML model quality was poor, and (4) perception that this type of study is an important thought exercise for clinicians. Overall, ML-based CDSSs have the potential to change treatment decisions in cancer management. However, meticulous development and validation of these systems as well as clinician training are required before deployment.

翻译：我们评估了临床医生在面对随机对照试验和机器学习模型结果时如何进行临床决策。为此，我们设计了一个临床决策支持系统，该系统展示了对12例多发性骨髓瘤患者的合成随机对照试验数据和机器学习模型得出的生存曲线及不良事件信息。我们开展了一项模拟环境下的介入性研究，以评估临床医生如何综合现有数据进行治疗决策。参与者被邀请参加后续开放式访谈，讨论其选择依据。当机器学习模型结果与随机对照试验结果一致时，医生对治疗选择的信心比仅获得随机对照试验结果时更高。当机器学习模型结果与随机对照试验结果不一致时，大多数医生在治疗选择中遵循了机器学习模型的建议。在医生获知模型训练和验证数据后，对机器学习模型的可靠认知度持续提升。后续访谈揭示了四大主题：(1) 参与者决策所用变量的差异性，(2) 机器学习模型相对于随机对照试验数据的认知优势，(3) 机器学习模型质量不佳时决策的不确定性，(4) 此类研究对临床医生具有重要思维训练价值的认知。总体而言，基于机器学习的临床决策支持系统具有改变癌症管理治疗决策的潜力。但在部署前，需对这些系统进行严谨开发与验证，并对临床医生进行充分培训。