Despite the remarkable success of Large Language Models (LLMs) in text understanding and generation, their potential for text clustering tasks remains underexplored. We observed that powerful closed-source LLMs provide good quality clusterings of entity sets but are not scalable due to the massive compute power required and the associated costs. Thus, we propose CACTUS (Context-Aware ClusTering with aUgmented triplet losS), a systematic approach that leverages open-source LLMs for efficient and effective supervised clustering of entity subsets, particularly focusing on text-based entities. Existing text clustering methods fail to effectively capture the context provided by the entity subset. Moreover, though there are several language modeling based approaches for clustering, very few are designed for the task of supervised clustering. This paper introduces a novel approach towards clustering entity subsets using LLMs by capturing context via a scalable inter-entity attention mechanism. We propose a novel augmented triplet loss function tailored for supervised clustering, which addresses the inherent challenges of directly applying the triplet loss to this problem. Furthermore, we introduce a self-supervised clustering task based on text augmentation techniques to improve the generalization of our model. For evaluation, we collect ground truth clusterings from a closed-source LLM and transfer this knowledge to an open-source LLM under the supervised clustering framework, allowing a faster and cheaper open-source model to perform the same task. Experiments on various e-commerce query and product clustering datasets demonstrate that our proposed approach significantly outperforms existing unsupervised and supervised baselines under various external clustering evaluation metrics.

翻译：尽管大语言模型（LLMs）在文本理解与生成领域取得了显著成功，但其在文本聚类任务中的潜力仍未被充分探索。我们观察到，强大的闭源LLMs虽能对实体集生成高质量聚类结果，但由于需要海量算力及相关成本，其可扩展性受限。为此，我们提出CACTUS（基于增强三元组损失的上下文感知聚类方法），这是一套系统化方案，通过利用开源LLMs实现对实体子集的高效监督聚类，尤其聚焦于文本型实体。现有文本聚类方法难以有效捕获实体子集提供的上下文信息。此外，尽管存在多种基于语言建模的聚类方法，但极少针对监督聚类任务设计。本文提出一种利用LLMs进行实体子集聚类的新方法，通过可扩展的跨实体注意力机制捕获上下文信息。我们设计了一种面向监督聚类的新型增强三元组损失函数，解决了将三元组损失直接应用于该问题的固有挑战。同时，引入基于文本增强技术的自监督聚类任务以提升模型泛化能力。为进行评估，我们从闭源LLM获取真实聚类标签，并基于监督聚类框架将该知识迁移至开源LLM，使更快速、经济的开源模型能够执行相同任务。在多项电商查询与商品聚类数据集上的实验表明，本文方法在各种外部聚类评价指标下均显著优于现有无监督与监督基线方法。