We present DiffExplainer, a novel framework that, leveraging language-vision models, enables multimodal global explainability. DiffExplainer employs diffusion models conditioned on optimized text prompts, synthesizing images that maximize class outputs and hidden features of a classifier, thus providing a visual tool for explaining decisions. Moreover, the analysis of generated visual descriptions allows for automatic identification of biases and spurious features, as opposed to traditional methods that often rely on manual intervention. The cross-modal transferability of language-vision models also enables the possibility to describe decisions in a more human-interpretable way, i.e., through text. We conduct comprehensive experiments, which include an extensive user study, demonstrating the effectiveness of DiffExplainer on 1) the generation of high-quality images explaining model decisions, surpassing existing activation maximization methods, and 2) the automated identification of biases and spurious features.

翻译：我们提出 DiffExplainer，一种新颖的框架，通过利用语言-视觉模型实现多模态全局可解释性。DiffExplainer 采用以优化文本提示为条件的扩散模型，合成能够最大化分类器类输出与隐藏特征的图像，从而为决策解释提供可视化工具。此外，与传统依赖人工干预的方法不同，对生成的视觉描述进行分析可实现偏差与虚假特征的自动识别。语言-视觉模型的跨模态可迁移性还使得能够以更符合人类理解的方式（即通过文本）描述决策。我们进行了全面的实验，包括一项大规模用户研究，证明了 DiffExplainer 在以下两方面的有效性：1）生成解释模型决策的高质量图像，超越现有激活最大化方法；2）自动识别偏差与虚假特征。