Large multimodal models (LMMs) combine unimodal encoders and large language models (LLMs) to perform multimodal tasks. Despite recent advancements towards the interpretability of these models, understanding internal representations of LMMs remains largely a mystery. In this paper, we present a novel framework for the interpretation of LMMs. We propose a dictionary learning based approach, applied to the representation of tokens. The elements of the learned dictionary correspond to our proposed concepts. We show that these concepts are well semantically grounded in both vision and text. Thus we refer to these as "multi-modal concepts". We qualitatively and quantitatively evaluate the results of the learnt concepts. We show that the extracted multimodal concepts are useful to interpret representations of test samples. Finally, we evaluate the disentanglement between different concepts and the quality of grounding concepts visually and textually. We will publicly release our code.

翻译：大型多模态模型（LMMs）通过结合单模态编码器与大型语言模型（LLMs）来执行多模态任务。尽管近年来这些模型的可解释性研究取得了进展，但理解LMMs的内部表征在很大程度上仍是一个未解之谜。本文提出一种新颖的LMMs解释框架。我们采用基于字典学习的方法，并将其应用于词元表征。学习所得字典的元素对应于我们提出的概念。研究表明，这些概念在视觉和文本模态中均具有良好的语义基础，因此我们将其称为“多模态概念”。我们对学习所得概念进行了定性与定量评估，证明提取的多模态概念能有效解释测试样本的表征。最后，我们从视觉和文本两个维度评估了不同概念间的解耦程度以及概念基础的质量。我们将公开相关代码。