Online forums encourage the exchange and discussion of different stances on many topics. Not only do they provide an opportunity to present one's own arguments, but may also gather a broad cross-section of others' arguments. However, the resulting long discussions are difficult to overview. This paper presents a novel unsupervised approach using large language models (LLMs) to generating indicative summaries for long discussions that basically serve as tables of contents. Our approach first clusters argument sentences, generates cluster labels as abstractive summaries, and classifies the generated cluster labels into argumentation frames resulting in a two-level summary. Based on an extensively optimized prompt engineering approach, we evaluate 19~LLMs for generative cluster labeling and frame classification. To evaluate the usefulness of our indicative summaries, we conduct a purpose-driven user study via a new visual interface called Discussion Explorer: It shows that our proposed indicative summaries serve as a convenient navigation tool to explore long discussions.
翻译:在线论坛鼓励围绕众多话题交换和讨论不同立场。这类平台不仅提供了陈述个人论点的机会,还能汇集来自他人的广泛观点。然而,由此产生的冗长讨论往往难以概览。本文提出了一种利用大型语言模型(LLM)的无监督新方法,用于生成长篇讨论的指示性摘要,其功能类似于目录。该方法首先对论点句子进行聚类,生成聚类标签作为抽象式摘要,接着将生成的聚类标签归入论证框架,最终形成双层摘要。基于广泛优化的提示工程方法,我们评估了19种大型语言模型在生成式聚类标签与框架分类任务上的表现。为验证指示性摘要的实用性,我们通过新型可视化界面“讨论探索者”开展了目标导向型用户研究:结果表明,本文提出的指示性摘要可作为便捷导航工具,有效辅助用户探索长篇讨论。