Transformer-based models have gained popularity in the field of natural language processing (NLP) and are extensively utilized in computer vision tasks and multi-modal models such as GPT4. This paper presents a novel method to enhance the explainability of Transformer-based image classification models. Our method aims to improve trust in classification results and empower users to gain a deeper understanding of the model for downstream tasks by providing visualizations of class-specific maps. We introduce two modules: the ``Relationship Weighted Out" and the ``Cut" modules. The ``Relationship Weighted Out" module focuses on extracting class-specific information from intermediate layers, enabling us to highlight relevant features. Additionally, the ``Cut" module performs fine-grained feature decomposition, taking into account factors such as position, texture, and color. By integrating these modules, we generate dense class-specific visual explainability maps. We validate our method with extensive qualitative and quantitative experiments on the ImageNet dataset. Furthermore, we conduct a large number of experiments on the LRN dataset, specifically designed for automatic driving danger alerts, to evaluate the explainability of our method in complex backgrounds. The results demonstrate a significant improvement over previous methods. Moreover, we conduct ablation experiments to validate the effectiveness of each module. Through these experiments, we are able to confirm the respective contributions of each module, thus solidifying the overall effectiveness of our proposed approach.

翻译：基于Transformer的模型在自然语言处理领域已获得广泛关注，并逐步应用于计算机视觉任务及GPT4等多模态模型。本文提出一种新颖方法，旨在增强基于Transformer的图像分类模型的可解释性。通过生成类别特定可视化图谱，该方法旨在提升对分类结果的信任度，并帮助用户更深入地理解模型以支持下游任务。我们引入两个模块："关系加权输出"模块与"剪切"模块。"关系加权输出"模块专注于从中间层提取类别特定信息，实现相关特征的增强；"剪切"模块则进行细粒度特征分解，综合考虑位置、纹理与颜色等因素。通过整合这些模块，我们生成密集的类别特定可视化可解释性图谱。我们在ImageNet数据集上开展了广泛的定性与定量实验以验证该方法。此外，针对自动驾驶危险预警场景，我们在LRN数据集上进行了大量实验，评估该方法在复杂背景下的可解释性。实验结果表明，该方法较现有方法有显著提升。同时，我们通过消融实验验证了各模块的有效性，确认了每个模块的独立贡献，从而夯实了所提方法的整体有效性。