In order to trust the predictions of a machine learning algorithm, it is necessary to understand the factors that contribute to those predictions. In the case of probabilistic and uncertainty-aware models, it is necessary to understand not only the reasons for the predictions themselves, but also the model's level of confidence in those predictions. In this paper, we show how existing methods in explainability can be extended to uncertainty-aware models and how such extensions can be used to understand the sources of uncertainty in a model's predictive distribution. In particular, by adapting permutation feature importance, partial dependence plots, and individual conditional expectation plots, we demonstrate that novel insights into model behaviour may be obtained and that these methods can be used to measure the impact of features on both the entropy of the predictive distribution and the log-likelihood of the ground truth labels under that distribution. With experiments using both synthetic and real-world data, we demonstrate the utility of these approaches in understanding both the sources of uncertainty and their impact on model performance.

翻译：为了信赖机器学习算法的预测结果，有必要理解影响这些预测的因素。对于概率型及不确定性感知模型而言，不仅需要理解预测本身的成因，还需了解模型对这些预测的置信水平。本文展示了如何将现有的可解释性方法扩展到不确定性感知模型，并利用这些扩展理解模型预测分布中不确定性的来源。具体而言，通过调整排列特征重要性、部分依赖图及个体条件期望图，我们证明可以获得关于模型行为的新见解，并利用这些方法衡量特征对预测分布熵以及该分布下真实标签对数似然的影响。通过合成数据与实际数据的实验，我们验证了这些方法在理解不确定性来源及其对模型性能影响方面的实用性。