Automatic fact verification has received significant attention recently. Contemporary automatic fact-checking systems focus on estimating truthfulness using numerical scores which are not human-interpretable. A human fact-checker generally follows several logical steps to verify a verisimilitude claim and conclude whether it is truthful or a mere masquerade. Popular fact-checking websites follow a common structure for fact categorization such as half true, half false, false, pants on fire, etc. Therefore, it is necessary to have an aspect-based (which part is true and which part is false) explainable system that can assist human fact-checkers in asking relevant questions related to a fact, which can then be validated separately to reach a final verdict. In this paper, we propose a 5W framework (who, what, when, where, and why) for question-answer-based fact explainability. To that end, we have gathered a semi-automatically generated dataset called FACTIFY-5WQA, which consists of 395, 019 facts along with relevant 5W QAs underscoring our major contribution to this paper. A semantic role labeling system has been utilized to locate 5Ws, which generates QA pairs for claims using a masked language model. Finally, we report a baseline QA system to automatically locate those answers from evidence documents, which can be served as the baseline for future research in this field. Lastly, we propose a robust fact verification system that takes paraphrased claims and automatically validates them. The dataset and the baseline model are available at https://github.com/ankuranii/acl-5W-QA.
翻译:自动事实核查技术近期受到广泛关注。当代自动化事实核查系统通常依赖数值评分评估真实性,但此类评分缺乏人类可解释性。人类核查员通常遵循若干逻辑步骤验证某声称的真实性,并判断其确凿可信或仅为虚假伪装。主流事实核查网站采用通用分类体系(如半真、半假、虚假、完全捏造等),因此亟需构建基于方面(即逐部分判定真伪)的可解释系统,辅助核查员围绕事实提出针对性问题并逐一验证,最终形成综合裁决。本文提出基于5W框架(何人、何事、何时、何地、为何)的问答式事实可解释方法。为此,我们构建了半自动生成数据集FACTIFY-5WQA,包含395,019条事实及其对应的5W问答对,此为本文核心贡献。我们利用语义角色标注系统定位5W要素,并基于掩码语言模型为每条主张生成问答对。最后,我们构建了基准问答系统,能够从证据文档中自动定位答案,为后续研究提供基线参照。此外,我们提出鲁棒性事实核查系统,可自动验证释义后的主张。数据集与基线模型已开源至https://github.com/ankuranii/acl-5W-QA。