Explainable AI methods facilitate the understanding of model behaviour, yet, small, imperceptible perturbations to inputs can vastly distort explanations. As these explanations are typically evaluated holistically, before model deployment, it is difficult to assess when a particular explanation is trustworthy. Some studies have tried to create confidence estimators for explanations, but none have investigated an existing link between uncertainty and explanation quality. We artificially simulate epistemic uncertainty in text input by introducing noise at inference time. In this large-scale empirical study, we insert different levels of noise perturbations and measure the effect on the output of pre-trained language models and different uncertainty metrics. Realistic perturbations have minimal effect on performance and explanations, yet masking has a drastic effect. We find that high uncertainty doesn't necessarily imply low explanation plausibility; the correlation between the two metrics can be moderately positive when noise is exposed during the training process. This suggests that noise-augmented models may be better at identifying salient tokens when uncertain. Furthermore, when predictive and epistemic uncertainty measures are over-confident, the robustness of a saliency map to perturbation can indicate model stability issues. Integrated Gradients shows the overall greatest robustness to perturbation, while still showing model-specific patterns in performance; however, this phenomenon is limited to smaller Transformer-based language models.

翻译：可解释人工智能方法有助于理解模型行为，然而输入中微小、不易察觉的扰动可能大幅扭曲解释结果。由于这些解释通常在模型部署前进行整体评估，因此难以判断特定解释的可信度。部分研究尝试为解释构建置信度估计器，但尚无研究探讨不确定性与解释质量之间存在的关联。我们通过在推理阶段引入噪声，人为模拟文本输入中的认知不确定性。在这项大规模实证研究中，我们插入不同级别的噪声扰动，并测量其对预训练语言模型输出及多种不确定性指标的影响。实际扰动对模型性能与解释的影响微乎其微，而掩码操作则产生剧烈效应。研究发现，高不确定性并不必然意味着低解释合理性；当训练过程中暴露噪声时，两个指标间的相关性可能呈现中等正向关联。这表明噪声增强模型在不确定时可能更擅长识别关键标记。此外，当预测不确定性与认知不确定性度量过度自信时，显著性图对扰动的鲁棒性可反映模型稳定性问题。积分梯度法对扰动展现出最高的整体鲁棒性，但仍呈现模型特异性性能模式；然而，该现象仅限于较小规模的基于Transformer语言模型。