Presenting high-level arguments is a crucial task for fostering participation in online societal discussions. Current argument summarization approaches miss an important facet of this task -- capturing diversity -- which is important for accommodating multiple perspectives. We introduce three aspects of diversity: those of opinions, annotators, and sources. We evaluate approaches to a popular argument summarization task called Key Point Analysis, which shows how these approaches struggle to (1) represent arguments shared by few people, (2) deal with data from various sources, and (3) align with subjectivity in human-provided annotations. We find that both general-purpose LLMs and dedicated KPA models exhibit this behavior, but have complementary strengths. Further, we observe that diversification of training data may ameliorate generalization. Addressing diversity in argument summarization requires a mix of strategies to deal with subjectivity.
翻译:呈现高层次的论点是促进在线社会讨论参与的关键任务。当前的论证摘要方法忽略了这一任务的重要方面——捕捉多样性——这对于容纳多种视角至关重要。我们引入多样性的三个维度:观点多样性、标注者多样性和来源多样性。我们评估了一种流行的论证摘要任务(关键点分析,Key Point Analysis)的方法,结果表明这些方法难以:(1)代表少数人共享的论点,(2)处理来自不同来源的数据,(3)与人类提供的标注中的主观性对齐。我们发现通用的大型语言模型(LLM)和专用关键点分析(KPA)模型均表现出这一行为,但具有互补优势。此外,我们观察到训练数据的多样化可能改善泛化能力。解决论证摘要中的多样性需要结合多种策略以应对主观性。