The argument sufficiency assessment task aims to determine if the premises of a given argument support its conclusion. To tackle this task, existing works often train a classifier on data annotated by humans. However, annotating data is laborious, and annotations are often inconsistent due to subjective criteria. Motivated by the definition of probability of sufficiency (PS) in the causal literature, we proposeCASA, a zero-shot causality-driven argument sufficiency assessment framework. PS measures how likely introducing the premise event would lead to the conclusion when both the premise and conclusion events are absent. To estimate this probability, we propose to use large language models (LLMs) to generate contexts that are inconsistent with the premise and conclusion and revise them by injecting the premise event. Experiments on two logical fallacy detection datasets demonstrate that CASA accurately identifies insufficient arguments. We further deploy CASA in a writing assistance application, and find that suggestions generated by CASA enhance the sufficiency of student-written arguments. Code and data are available at https://github.com/xxxiaol/CASA.
翻译:论证充分性评估任务旨在判断给定论证的前提是否支持其结论。为解决此任务,现有研究通常基于人工标注数据训练分类器。然而,数据标注工作繁重,且因主观标准差异常导致标注不一致。受因果文献中充分性概率(Probability of Sufficiency, PS)定义的启发,我们提出CASA——一种零样本因果驱动论证充分性评估框架。PS衡量当前提事件和结论事件均未发生时,引入前提事件导致结论发生的可能性。为估计该概率,我们提议使用大型语言模型(LLMs)生成与前提和结论不一致的上下文,并通过注入前提事件对其进行修订。在两个逻辑谬误检测数据集上的实验表明,CASA能准确识别不充分的论证。我们进一步将CASA部署于写作辅助应用中,发现其生成的建议能有效增强学生撰写论证的充分性。代码与数据见https://github.com/xxxiaol/CASA。