Image clustering aims to group images in an unsupervised fashion. Traditional methods focus on knowledge from visual space, making it difficult to distinguish between visually similar but semantically different classes. Recent advances in vision-language models enable the use of textual knowledge to enhance image clustering. However, most existing methods rely on coarse class labels or simple nouns, overlooking the rich conceptual and attribute-level semantics embedded in textual space. In this paper, we propose a knowledge-enhanced clustering (KEC) method that constructs a hierarchical concept-attribute structured knowledge with the help of large language models (LLMs) to guide clustering. Specifically, we first condense redundant textual labels into abstract concepts and then automatically extract discriminative attributes for each single concept and similar concept pairs, via structured prompts to LLMs. This knowledge is instantiated for each input image to achieve the knowledge-enhanced features. The knowledge-enhanced features with original visual features are adapted to various downstream clustering algorithms. We evaluate KEC on 20 diverse datasets, showing consistent improvements across existing methods using additional textual knowledge. KEC without training outperforms zero-shot CLIP on 14 out of 20 datasets. Furthermore, the naive use of textual knowledge may harm clustering performance, while KEC provides both accuracy and robustness.

翻译：图像聚类旨在以无监督方式对图像进行分组。传统方法主要依赖视觉空间中的知识，难以区分视觉相似但语义不同的类别。近期视觉-语言模型的进展使得利用文本知识增强图像聚类成为可能。然而，现有方法大多依赖粗粒度类别标签或简单名词，忽视了文本空间中蕴含的丰富概念级与属性级语义。本文提出一种知识增强聚类（KEC）方法，借助大语言模型（LLMs）构建层级化概念-属性结构化知识以指导聚类。具体而言，我们首先将冗余文本标签压缩为抽象概念，随后通过结构化提示词对LLMs进行引导，自动提取每个单一概念及相似概念对的判别性属性。这些知识被实例化至每个输入图像，从而获得知识增强特征。知识增强特征与原始视觉特征可适配至多种下游聚类算法。我们在20个多样化数据集上评估KEC，表明其能通过额外文本知识持续提升现有方法性能。其中，无训练的KEC在20个数据集中有14个超越零样本CLIP。此外，文本知识的简单使用可能损害聚类性能，而KEC兼具准确性与鲁棒性。