The performance of convolutional neural networks has continued to improve over the last decade. At the same time, as model complexity grows, it becomes increasingly more difficult to explain model decisions. Such explanations may be of critical importance for reliable operation of human-machine pairing setups, or for model selection when the "best" model among many equally-accurate models must be established. Saliency maps represent one popular way of explaining model decisions by highlighting image regions models deem important when making a prediction. However, examining salience maps at scale is not practical. In this paper, we propose five novel methods of leveraging model salience to explain a model behavior at scale. These methods ask: (a) what is the average entropy for a model's salience maps, (b) how does model salience change when fed out-of-set samples, (c) how closely does model salience follow geometrical transformations, (d) what is the stability of model salience across independent training runs, and (e) how does model salience react to salience-guided image degradations. To assess the proposed measures on a concrete and topical problem, we conducted a series of experiments for the task of synthetic face detection with two types of models: those trained traditionally with cross-entropy loss, and those guided by human salience when training to increase model generalizability. These two types of models are characterized by different, interpretable properties of their salience maps, which allows for the evaluation of the correctness of the proposed measures. We offer source codes for each measure along with this paper.

翻译：卷积神经网络的性能在过去十年中持续提升。与此同时，随着模型复杂度的增长，解释模型决策变得越来越困难。这种人机配对场景的可靠运行，或在众多精度相当的模型中确定"最佳"模型时，此类解释可能至关重要。显著性图是一种流行的模型决策解释方法，通过高亮模型在预测时认为重要的图像区域来呈现其决策依据。然而，大规模分析显著性图并不可行。本文提出五种利用模型显著性进行规模化行为解释的新方法，分别探究：(a)模型显著性图的平均熵值，(b)输入非训练集样本时显著性的变化模式，(c)显著性对几何变换的跟踪精度，(d)独立训练轮次间显著性的稳定性，以及(e)显著性对引导式图像退化操作的响应特性。为在具体且具前沿性的问题上评估所提方法，我们以合成人脸检测任务为场景开展系列实验，涉及两类模型：传统交叉熵损失训练的模型，以及通过人类显著性引导训练以提升泛化能力的模型。这两类模型的显著性图具有不同可解释特征，可用于验证所提方法的正确性。本文随附所有方法的源代码。