Explainable AI has brought transparency into complex ML blackboxes, enabling, in particular, to identify which features these models use for their predictions. So far, the question of explaining predictive uncertainty, i.e. why a model 'doubts', has been scarcely studied. Our investigation reveals that predictive uncertainty is dominated by second-order effects, involving single features or product interactions between them. We contribute a new method for explaining predictive uncertainty based on these second-order effects. Computationally, our method reduces to a simple covariance computation over a collection of first-order explanations. Our method is generally applicable, allowing for turning common attribution techniques (LRP, Gradient x Input, etc.) into powerful second-order uncertainty explainers, which we call CovLRP, CovGI, etc. The accuracy of the explanations our method produces is demonstrated through systematic quantitative evaluations, and the overall usefulness of our method is demonstrated via two practical showcases.

翻译：可解释人工智能已为复杂的机器学习黑箱带来了透明度，使我们尤其能够识别这些模型用于预测的特征。迄今为止，关于解释预测不确定性（即模型为何“犹豫”）的研究仍很少。我们的研究表明，预测不确定性主要由二阶效应主导，这些效应涉及单一特征或特征之间的乘积交互。我们提出了一种基于这些二阶效应的新方法来解释预测不确定性。在计算上，我们的方法简化为对一阶解释集合进行简单的协方差计算。该方法具有普遍适用性，可将常见的归因技术（如LRP、Gradient×Input等）转化为强大的二阶不确定性解释器，我们将其命名为CovLRP、CovGI等。通过系统的定量评估，我们证明了该方法生成的解释的准确性，并通过两个实际案例展示了其整体实用性。