The task of rumour verification in social media concerns assessing the veracity of a claim on the basis of conversation threads that result from it. While previous work has focused on predicting a veracity label, here we reformulate the task to generate model-centric, free-text explanations of a rumour's veracity. We follow an unsupervised approach by first utilising post-hoc explainability methods to score the most important posts within a thread and then we use these posts to generate informative explanatory summaries by employing template-guided summarisation. To evaluate the informativeness of the explanatory summaries, we exploit the few-shot learning capabilities of a large language model (LLM). Our experiments show that LLMs can have similar agreement to humans in evaluating summaries. Importantly, we show that explanatory abstractive summaries are more informative and better reflect the predicted rumour veracity than just using the highest ranking posts in the thread.
翻译:社交媒体中的谣言验证任务涉及根据由声明引发的对话线程评估其真实性。以往研究主要集中于预测真实性标签,本文则重新构述该任务,以生成以模型为中心的谣言真实性自由文本解释。我们采用无监督方法:首先利用事后可解释性方法对线程中最关键的帖子进行评分,随后通过模板引导式摘要技术,基于这些帖子生成信息丰富的解释性摘要。为评估解释性摘要的信息量,我们借助大语言模型(LLM)的少样本学习能力。实验表明,LLM在评估摘要方面与人类具有相似的一致性。更重要的是,我们证明:相比仅使用线程中排名最高的帖子,解释性抽象化摘要更能反映预测的谣言真实性且信息量更丰富。