Fighting misinformation is a challenging, yet crucial, task. Despite the growing number of experts being involved in manual fact-checking, this activity is time-consuming and cannot keep up with the ever-increasing amount of Fake News produced daily. Hence, automating this process is necessary to help curb misinformation. Thus far, researchers have mainly focused on claim veracity classification. In this paper, instead, we address the generation of justifications (textual explanation of why a claim is classified as either true or false) and benchmark it with novel datasets and advanced baselines. In particular, we focus on summarization approaches over unstructured knowledge (i.e. news articles) and we experiment with several extractive and abstractive strategies. We employed two datasets with different styles and structures, in order to assess the generalizability of our findings. Results show that in justification production summarization benefits from the claim information, and, in particular, that a claim-driven extractive step improves abstractive summarization performances. Finally, we show that although cross-dataset experiments suffer from performance degradation, a unique model trained on a combination of the two datasets is able to retain style information in an efficient manner.
翻译:打击虚假信息是一项充满挑战但至关重要的任务。尽管越来越多的专家参与人工事实核查,但这一过程耗时费力,难以跟上每日剧增的假新闻数量。因此,自动化该流程对于遏制虚假信息传播十分必要。迄今为止,研究者主要聚焦于主张真实性分类。而在本文中,我们转而探讨论证生成(即对主张被判定为真或假提供文本解释)的问题,并基于新型数据集和先进基线方法进行基准测试。具体而言,我们聚焦于对非结构化知识(如新闻文章)的摘要生成方法,并实验了多种抽取式与生成式策略。为评估研究结果的泛化性,我们采用了两种风格与结构各异的数据集。结果表明,在论证生成中,摘要生成可从主张信息中获益,尤其当主张驱动的抽取步骤能够提升生成式摘要的性能。最后,尽管跨数据集实验存在性能下降问题,但基于双数据集联合训练的单一模型能高效保留风格信息。