We investigate the CLIP image encoder by analyzing how individual model components affect the final representation. We decompose the image representation as a sum across individual image patches, model layers, and attention heads, and use CLIP's text representation to interpret the summands. Interpreting the attention heads, we characterize each head's role by automatically finding text representations that span its output space, which reveals property-specific roles for many heads (e.g. location or shape). Next, interpreting the image patches, we uncover an emergent spatial localization within CLIP. Finally, we use this understanding to remove spurious features from CLIP and to create a strong zero-shot image segmenter. Our results indicate that a scalable understanding of transformer models is attainable and can be used to repair and improve models.

翻译：我们通过分析单个模型组件对最终表示的影响来研究CLIP图像编码器。将图像表示分解为各图像块、模型层和注意力头的贡献之和，并利用CLIP的文本表示对这些分量进行解读。在解释注意力头时，我们通过自动寻找能够跨越其输出空间的文本表示来刻画每个头的功能，这一方法揭示了许多头具有特定属性的作用（例如位置或形状）。接下来，通过解读图像块，我们发现了CLIP中涌现的空间定位能力。最后，利用这些理解，我们移除了CLIP中的虚假特征，并构建了一个强大的零样本图像分割器。我们的结果表明，对Transformer模型的可扩展性理解是可行的，并可被用于修复和改进模型。