Saliency Maps (SMs) have been extensively used to interpret deep learning models decision by highlighting the features deemed relevant by the model. They are used on highly nonlinear problems, where linear feature selection (FS) methods fail at highlighting relevant explanatory variables. However, the reliability of gradient-based feature attribution methods such as SM has mostly been only qualitatively (visually) assessed, and quantitative benchmarks are currently missing, partially due to the lack of a definite ground truth on image data. Concerned about the apophenic biases introduced by visual assessment of these methods, in this paper we propose a synthetic quantitative benchmark for Neural Networks (NNs) interpretation methods. For this purpose, we built synthetic datasets with nonlinearly separable classes and increasing number of decoy (random) features, illustrating the challenge of FS in high-dimensional settings. We also compare these methods to conventional approaches such as mRMR or Random Forests. Our results show that our simple synthetic datasets are sufficient to challenge most of the benchmarked methods. TreeShap, mRMR and LassoNet are the best performing FS methods. We also show that, when quantifying the relevance of a few non linearly-entangled predictive features diluted in a large number of irrelevant noisy variables, neural network-based FS and interpretation methods are still far from being reliable.

翻译：显著性图（Saliency Maps）已被广泛用于通过突出模型认为相关的特征来解释深度学习模型的决策。它们应用于高度非线性问题，而传统的线性特征选择方法在这些问题中无法有效识别相关解释变量。然而，基于梯度的特征归因方法（如显著性图）的可靠性大多仅通过定性（视觉）评估，目前尚缺乏定量基准，这在一定程度上是由于图像数据缺乏明确的真实基准。针对这些方法在视觉评估中可能引入的幻想性偏见，本文提出了一种用于神经网络解释方法的合成定量基准。为此，我们构建了具有非线性可分类别和递增数量干扰（随机）特征的合成数据集，以展示高维特征选择中的挑战。我们还将这些方法与传统的特征选择方法（如mRMR或随机森林）进行了比较。结果表明，我们简单的合成数据集足以挑战大多数被基准测试的方法。TreeShap、mRMR和LassoNet是表现最佳的特征选择方法。我们还表明，在量化稀释于大量无关噪声变量中的少数非线性纠缠预测特征的相关性时，基于神经网络的特征选择与解释方法仍远不可靠。