In this paper, we introduce an explainable algorithm designed from a multi-modal foundation model, that performs fast and explainable image classification. Drawing inspiration from CLIP-based Concept Bottleneck Models (CBMs), our method creates a latent space where each neuron is linked to a specific word. Observing that this latent space can be modeled with simple distributions, we use a Mixture of Gaussians (MoG) formalism to enhance the interpretability of this latent space. Then, we introduce CLIP-QDA, a classifier that only uses statistical values to infer labels from the concepts. In addition, this formalism allows for both local and global explanations. These explanations come from the inner design of our architecture, our work is part of a new family of greybox models, combining performances of opaque foundation models and the interpretability of transparent models. Our empirical findings show that in instances where the MoG assumption holds, CLIP-QDA achieves similar accuracy with state-of-the-art methods CBMs. Our explanations compete with existing XAI methods while being faster to compute.

翻译：在本文中，我们提出了一种基于多模态基础模型设计的可解释算法，能够实现快速且可解释的图像分类。受基于CLIP的概念瓶颈模型（CBMs）启发，我们的方法构建了一个潜在空间，其中每个神经元与特定词语相关联。观测到该潜在空间可通过简单分布进行建模后，我们采用混合高斯模型（MoG）形式化方法来增强该潜在空间的可解释性。进而引入CLIP-QDA分类器，该分类器仅使用统计值从概念中推断标签。此外，这种形式化方法支持局部与全局解释，这些解释源于我们架构的内在设计。我们的工作属于新型灰盒模型家族，兼具不透明基础模型的性能与透明模型的可解释性。实验结果表明，在MoG假设成立的情况下，CLIP-QDA能够达到与现有最先进的CBMs方法相当的准确率。其解释能力可与现有可解释人工智能（XAI）方法相媲美，同时计算速度更快。