Deep learning models for learning analytics have become increasingly popular over the last few years; however, these approaches are still not widely adopted in real-world settings, likely due to a lack of trust and transparency. In this paper, we tackle this issue by implementing explainable AI methods for black-box neural networks. This work focuses on the context of online and blended learning and the use case of student success prediction models. We use a pairwise study design, enabling us to investigate controlled differences between pairs of courses. Our analyses cover five course pairs that differ in one educationally relevant aspect and two popular instance-based explainable AI methods (LIME and SHAP). We quantitatively compare the distances between the explanations across courses and methods. We then validate the explanations of LIME and SHAP with 26 semi-structured interviews of university-level educators regarding which features they believe contribute most to student success, which explanations they trust most, and how they could transform these insights into actionable course design decisions. Our results show that quantitatively, explainers significantly disagree with each other about what is important, and qualitatively, experts themselves do not agree on which explanations are most trustworthy. All code, extended results, and the interview protocol are provided at https://github.com/epfl-ml4ed/trusting-explainers.

翻译：近年来，用于学习分析的深度学习模型日益普及；然而，这些方法在实际场景中仍未得到广泛应用，这很可能源于缺乏信任与透明度。本文通过为黑箱神经网络实现可解释人工智能方法来解决这一问题。本研究聚焦在线与混合式学习场景，以及学生成功预测模型的应用案例。我们采用配对研究设计，从而能够探究课程对之间的受控差异。分析涵盖五组在教育相关维度上存在差异的课程对，并采用两种流行的基于实例的可解释人工智能方法（LIME和SHAP）。我们定量比较了跨课程与方法间解释的差异。随后，通过26次对大学教师的半结构化访谈，验证了LIME和SHAP的解释，访谈涉及哪些特征最有助于学生成功、教师最信任哪种解释，以及如何将这些见解转化为可行的课程设计决策。结果表明：定量上，不同解释方法对重要性要素的判定存在显著分歧；定性上，专家自身对最可信的解释也未达成共识。所有代码、扩展结果及访谈协议均发布于https://github.com/epfl-ml4ed/trusting-explainers。