Whilst fact verification has attracted substantial interest in the natural language processing community, verifying misinforming statements against data visualizations such as charts has so far been overlooked. Charts are commonly used in the real-world to summarize and communicate key information, but they can also be easily misused to spread misinformation and promote certain agendas. In this paper, we introduce ChartCheck, a novel, large-scale dataset for explainable fact-checking against real-world charts, consisting of 1.7k charts and 10.5k human-written claims and explanations. We systematically evaluate ChartCheck using vision-language and chart-to-table models, and propose a baseline to the community. Finally, we study chart reasoning types and visual attributes that pose a challenge to these models

翻译：尽管事实核查在自然语言处理领域引起了广泛关注，但针对数据可视化（如图表）的虚假陈述验证问题至今仍未得到充分研究。图表在现实世界中常被用于总结和传达关键信息，但也容易被滥用以传播错误信息和推动特定议程。本文提出了ChartCheck——一个面向真实世界图表的新型大规模可解释事实核查数据集，包含1.7k张图表及10.5k条人工撰写的声明与解释。我们通过视觉-语言模型和图表-表格模型对ChartCheck进行了系统性评估，并为学界提供了基准方法。最后，我们研究了给这些模型带来挑战的图表推理类型及视觉属性。