We propose WIBA, a novel framework and suite of methods that enable the comprehensive understanding of "What Is Being Argued" across contexts. Our approach develops a comprehensive framework that detects: (a) the existence, (b) the topic, and (c) the stance of an argument, correctly accounting for the logical dependence among the three tasks. Our algorithm leverages the fine-tuning and prompt-engineering of Large Language Models. We evaluate our approach and show that it performs well in all the three capabilities. First, we develop and release an Argument Detection model that can classify a piece of text as an argument with an F1 score between 79% and 86% on three different benchmark datasets. Second, we release a language model that can identify the topic being argued in a sentence, be it implicit or explicit, with an average similarity score of 71%, outperforming current naive methods by nearly 40%. Finally, we develop a method for Argument Stance Classification, and evaluate the capability of our approach, showing it achieves a classification F1 score between 71% and 78% across three diverse benchmark datasets. Our evaluation demonstrates that WIBA allows the comprehensive understanding of What Is Being Argued in large corpora across diverse contexts, which is of core interest to many applications in linguistics, communication, and social and computer science. To facilitate accessibility to the advancements outlined in this work, we release WIBA as a free open access platform (wiba.dev).
翻译:我们提出WIBA,一种新颖的框架和方法套件,能够跨上下文全面理解“争论的是什么”。我们的方法开发了一个综合框架,可检测论证的(a)存在性、(b)主题以及(c)立场,并正确考虑了这三项任务之间的逻辑依赖关系。该算法利用了大语言模型的微调和提示工程。我们评估了该方法,并表明它在所有三项能力上均表现良好。首先,我们开发并发布了论证检测模型,该模型在三个不同基准数据集上能以79%至86%的F1分数将文本片段分类为论证。其次,我们发布了一个语言模型,能识别句子中争论的主题(无论是隐式还是显式),平均相似度得分为71%,比当前朴素方法高出近40%。最后,我们开发了一种论证立场分类方法,并评估了该方法的性能,表明其在三个不同基准数据集上实现了71%至78%的分类F1分数。我们的评估表明,WIBA能够全面理解大规模语料库中跨上下文的“争论的是什么”,这对语言学、传播学、社会科学和计算机科学中的许多应用具有核心意义。为方便获取本研究提出的进展,我们将WIBA作为免费开放访问平台发布(wiba.dev)。