Credibility signals represent a wide range of heuristics that are typically used by journalists and fact-checkers to assess the veracity of online content. Automating the task of credibility signal extraction, however, is very challenging as it requires high-accuracy signal-specific extractors to be trained, while there are currently no sufficiently large datasets annotated with all credibility signals. This paper investigates whether large language models (LLMs) can be prompted effectively with a set of 18 credibility signals to produce weak labels for each signal. We then aggregate these potentially noisy labels using weak supervision in order to predict content veracity. We demonstrate that our approach, which combines zero-shot LLM credibility signal labeling and weak supervision, outperforms state-of-the-art classifiers on two misinformation datasets without using any ground-truth labels for training. We also analyse the contribution of the individual credibility signals towards predicting content veracity, which provides new valuable insights into their role in misinformation detection.

翻译：可信度信号涵盖了新闻工作者和事实核查员通常用于评估在线内容真实性的一系列启发式规则。然而，自动化提取这些可信度信号的任务极具挑战性，因为这需要针对每个信号训练高精度的专门提取器，而目前尚不存在覆盖所有可信度信号且规模足够大的标注数据集。本文探究能否通过一组18个可信度信号有效提示大语言模型（LLMs），使其为每个信号生成弱标签。随后，我们利用弱监督方法聚合这些可能包含噪声的标签，以预测内容的真实性。实验表明，我们提出的将零样本LLM可信度信号标注与弱监督相结合的方法，在未使用任何真实标签进行训练的情况下，在两个错误信息数据集上的表现优于现有最优分类器。我们还分析了单个可信度信号对预测内容真实性的贡献，这为理解它们在错误信息检测中的作用提供了新的重要见解。