In modern machine learning, the trend of harnessing self-supervised learning to derive high-quality representations without label dependency has garnered significant attention. However, the absence of label information, coupled with the inherently high-dimensional nature, improves the difficulty for the interpretation of learned representations. Consequently, indirect evaluations become the popular metric for evaluating the quality of these features, leading to a biased validation of the learned representation rationale. To address these challenges, we introduce a novel approach termed Concept-based Explainable Image Representation (CEIR). Initially, using the Concept-based Model (CBM) incorporated with pretrained CLIP and concepts generated by GPT-4, we project input images into a concept vector space. Subsequently, a Variational Autoencoder (VAE) learns the latent representation from these projected concepts, which serves as the final image representation. Due to the capability of the representation to encapsulate high-level, semantically relevant concepts, the model allows for attributions to a human-comprehensible concept space. This not only enhances interpretability but also preserves the robustness essential for downstream tasks. For instance, our method exhibits state-of-the-art unsupervised clustering performance on benchmarks such as CIFAR10, CIFAR100, and STL10. Furthermore, capitalizing on the universality of human conceptual understanding, CEIR can seamlessly extract the related concept from open-world images without fine-tuning. This offers a fresh approach to automatic label generation and label manipulation.

翻译：在现代机器学习中，利用自监督学习在无需标签依赖的情况下获得高质量表示的趋势已引起广泛关注。然而，标签信息的缺失，加之表示本身固有的高维特性，增加了对所学表示进行解释的难度。因此，间接评估成为衡量这些特征质量的常用指标，导致对所学表示原理的验证存在偏差。为应对这些挑战，我们提出了一种名为基于概念的可解释图像表示（CEIR）的新方法。首先，结合使用预训练CLIP的概念模型（CBM）以及由GPT-4生成的概念，我们将输入图像投影到概念向量空间中。随后，变分自编码器（VAE）从这些投影概念中学习潜在表示，并作为最终的图像表示。由于该表示能够封装高层语义相关的概念，模型可将归因映射到人类可理解的概念空间。这不仅增强了可解释性，还保留了下游任务所需的鲁棒性。例如，我们的方法在CIFAR10、CIFAR100和STL10等基准测试中展现出最先进的无监督聚类性能。此外，利用人类概念理解的普适性，CEIR无需微调即可从开放世界图像中无缝提取相关概念，为自动标签生成和标签操作提供了一种全新思路。