The lack of transparency and explainability hinders the clinical adoption of Machine learning (ML) algorithms. While explainable artificial intelligence (XAI) methods have been proposed, little research has focused on the agreement between these methods and expert clinical knowledge. This study applies current state-of-the-art explainability methods to clinical decision support algorithms developed for Electronic Medical Records (EMR) data to analyse the concordance between these factors and discusses causes for identified discrepancies from a clinical and technical perspective. Important factors for achieving trustworthy XAI solutions for clinical decision support are also discussed.

翻译：机器学习算法缺乏透明度和可解释性阻碍了其在临床中的应用。尽管可解释人工智能方法已被提出，但针对这些方法与临床专家知识之间一致性的研究仍显不足。本研究将当前最先进的解释性方法应用于基于电子病历数据开发的临床决策支持算法，分析这些因素之间的一致性，并从临床和技术角度探讨已识别差异的成因。同时，本文还讨论了为实现临床决策支持中可信赖的可解释人工智能解决方案所需的关键因素。