Although point cloud models have gained significant improvements in prediction accuracy over recent years, their trustworthiness is still not sufficiently investigated. In terms of global explainability, Activation Maximization (AM) techniques in the image domain are not directly transplantable due to the special structure of the point cloud models. Existing studies exploit generative models to yield global explanations that can be perceived by humans. However, the opacity of the generative models themselves and the introduction of additional priors call into question the plausibility and fidelity of the explanations. In this work, we demonstrate that when the classifier predicts different types of instances, the intermediate layer activations are differently activated, known as activation flows. Based on this property, we propose an activation flow-based AM method that generates global explanations that can be perceived without incorporating any generative model. Furthermore, we reveal that AM based on generative models fails the sanity checks and thus lack of fidelity. Extensive experiments show that our approach dramatically enhances the perceptibility of explanations compared to other AM methods that are not based on generative models. Our code is available at: https://github.com/Explain3D/FlowAM
翻译:摘要:尽管近年来点云模型在预测精度上取得了显著提升,但其可信度研究仍不充分。在全局可解释性方面,图像域中的激活最大化(AM)技术因点云模型的特殊结构而无法直接移植。现有研究利用生成模型产生人类可感知的全局解释,但生成模型自身的不透明性以及额外先验的引入,使这些解释的合理性与保真度受到质疑。本文证明,当分类器预测不同类别的实例时,中间层激活会呈现差异化响应,即激活流。基于这一特性,我们提出一种基于激活流的AM方法,无需引入任何生成模型即可生成可感知的全局解释。此外,我们揭示基于生成模型的AM方法无法通过健全性检验,因此缺乏保真度。大量实验表明,与其他非生成式AM方法相比,本方法显著提升了解释的可感知性。我们的代码发布在:https://github.com/Explain3D/FlowAM