Unsupervised clustering is widely used to explore large corpora, but existing formulations neither consider the users' goals nor explain clusters' meanings. We propose a new task formulation, "Goal-Driven Clustering with Explanations" (GoalEx), which represents both the goal and the explanations as free-form language descriptions. For example, to categorize the errors made by a summarization system, the input to GoalEx is a corpus of annotator-written comments for system-generated summaries and a goal description "cluster the comments based on why the annotators think the summary is imperfect.''; the outputs are text clusters each with an explanation ("this cluster mentions that the summary misses important context information."), which relates to the goal and precisely explain which comments should (not) belong to a cluster. To tackle GoalEx, we prompt a language model with "[corpus subset] + [goal] + Brainstorm a list of explanations each representing a cluster."; then we classify whether each sample belongs to a cluster based on its explanation; finally, we use integer linear programming to select a subset of candidate clusters to cover most samples while minimizing overlaps. Under both automatic and human evaluation on corpora with or without labels, our method produces more accurate and goal-related explanations than prior methods. We release our data and implementation at https://github.com/ZihanWangKi/GoalEx.
翻译:无监督聚类广泛应用于探索大规模语料库,但现有方法既未考虑用户目标,也未解释聚类含义。我们提出新任务范式“目标驱动可解释聚类”(GoalEx),将目标和解释均表示为自由形式的语言描述。例如,为分类摘要系统产生的错误,GoalEx的输入是由标注员针对系统生成摘要撰写的评论文本及目标描述“根据标注员认为摘要不完美的原因对评论进行聚类”;输出是每个文本簇及其解释(如“该簇提到摘要遗漏了关键上下文信息”),这些解释与目标相关,并精确说明哪些评论应(或不应)归入该簇。为求解GoalEx,我们向语言模型输入“[语料子集] + [目标] + 头脑风暴生成代表各簇的解释列表”;随后基于解释判断每个样本是否属于对应簇;最后通过整数线性规划选择候选簇子集,在覆盖多数样本的同时最小化重叠。在带标签/无标签语料上的自动评估和人工评估中,我们的方法生成比先前方法更准确且与目标相关的解释。相关数据和实现已发布至 https://github.com/ZihanWangKi/GoalEx。