Pre-training of neural networks has recently revolutionized the field of Natural Language Processing (NLP) and has before demonstrated its effectiveness in computer vision. At the same time, advances around the detection of fake news were mainly driven by the context-based paradigm, where different types of signals (e.g. from social media) form graph-like structures that hold contextual information apart from the news article to classify. We propose to merge these two developments by applying pre-training of Graph Neural Networks (GNNs) in the domain of context-based fake news detection. Our experiments provide an evaluation of different pre-training strategies for graph-based misinformation detection and demonstrate that transfer learning does currently not lead to significant improvements over training a model from scratch in the domain. We argue that a major current issue is the lack of suitable large-scale resources that can be used for pre-training.

翻译：神经网络预训练技术近期革新了自然语言处理领域，此前已在计算机视觉中展现出卓越效能。与此同时，假新闻检测的进展主要受上下文驱动范式推动——该范式通过社交媒体等信号源构建图结构信息，在待分类新闻文章之外保留上下文特征。本研究提出将图神经网络预训练应用于上下文假新闻检测领域，融合这两大技术趋势。实验评估了面向图结构虚假信息检测的多种预训练策略，结果表明当前迁移学习在该领域并未显著优于从零开始训练的模型。我们认为当前的核心障碍在于缺乏适用于预训练的大规模数据资源。