As deep learning has achieved state-of-the-art performance for many tasks of EEG-based BCI, many efforts have been made in recent years trying to understand what have been learned by the models. This is commonly done by generating a heatmap indicating to which extent each pixel of the input contributes to the final classification for a trained model. Despite the wide use, it is not yet understood to which extent the obtained interpretation results can be trusted and how accurate they can reflect the model decisions. In order to fill this research gap, we conduct a study to evaluate different deep interpretation techniques quantitatively on EEG datasets. The results reveal the importance of selecting a proper interpretation technique as the initial step. In addition, we also find that the quality of the interpretation results is inconsistent for individual samples despite when a method with an overall good performance is used. Many factors, including model structure and dataset types, could potentially affect the quality of the interpretation results. Based on the observations, we propose a set of procedures that allow the interpretation results to be presented in an understandable and trusted way. We illustrate the usefulness of our method for EEG-based BCI with instances selected from different scenarios.

翻译：随着深度学习在EEG-based BCI的许多任务中实现了最先进的性能，近年来许多努力已致力于理解模型学习到了什么。这通常通过生成热图来完成，以指示哪些输入的像素对训练模型的最终分类有多大的贡献。尽管被广泛使用，但尚未理解所得到的解释结果在多大程度上可以信任，以及它们如何准确反映模型决策。为了填补这一研究空白，我们进行了一项研究，定量评估了不同的深度解释技术在EEG数据集上的表现。结果揭示了选择合适的解释技术作为起始步骤的重要性。此外，我们还发现，尽管使用了总体表现良好的方法，但个体样本的解释结果质量存在不一致性。许多因素，包括模型结构和数据集类型，都有可能影响解释结果的质量。基于这些观察结果，我们提出了一系列程序，使解释结果在可理解和可信的方式下呈现。我们选取不同情景下的实例，说明了我们的方法在基于EEG的BCI中的有用性。