Enthymemes, arguments with unstated premises or conclusions, are pervasive in persuasive discourse, yet their annotation remains notoriously subjective. We present a resource of 1,482 tweets from politically controversial discourse, annotated by five annotators for the presence of enthymemes and their argument structure, designed to study label variation. We first revisit the definition of enthymemes and propose annotation guidelines anchored in Walton's argumentation schemes, offering a structured and constrained approach that nonetheless preserves room for the interpretive nature of the task. This contrasts with past resources, which tend to eliminate disagreement, obscuring its sources and preventing investigation of its potential benefits for model performance. We further propose a complexity analysis of the task, identifying where annotation imposes high cognitive load and may give rise to inconsistent annotation. Our preliminary experiments show that models trained on annotator disagreement outperform models trained on hard majority-vote labels. We close by reflecting on how structural openness in enthymeme definitions and guidelines enables the study of variation in subjective inferential processes for future resources and downstream NLP applications concerned with human inference.
翻译:省略式论证(遗漏前提或结论的论证)在说服性话语中普遍存在,但其标注主观性极高。我们构建了一个包含1,482条来自政治争议话语推文的资源集,由五位标注者针对省略式论证的存在及其论证结构进行标注,旨在研究标签变异。首先重新审视了省略式论证的定义,并基于Walton论证图式提出标注指南,通过结构化约束方法保留任务解释性特征。这与既往资源不同——它们倾向于消除标注分歧,掩盖分歧来源并阻碍探究其潜在益处。进一步提出任务复杂度分析,识别标注过程中认知负荷较高的环节及潜在不一致根源。初步实验表明,基于标注者分歧训练的模型性能优于硬多数投票标签训练的模型。最后反思如何通过省略式论证定义与指南的结构开放性,为未来涉及人类推理的资源与下游NLP应用研究主观推理过程的变异提供支持。