Identifying when and where a news image was taken is crucial for journalists and forensic experts to produce credible stories and debunk misinformation. While many existing methods rely on reverse image search (RIS) engines, these tools often fail to return results, thereby limiting their practical applicability. In this work, we address the challenging scenario where RIS evidence is unavailable. We introduce NewsRECON, a method that links images to relevant news articles to infer their date and location from article metadata. NewsRECON leverages a corpus of over 90,000 articles and integrates: (1) a bi-encoder for retrieving event-relevant articles; (2) two cross-encoders for reranking articles by location and event consistency. Experiments on the TARA and 5Pils-OOC show that NewsRECON outperforms prior work and can be combined with a multimodal large language model to achieve new SOTA results in the absence of RIS evidence. We make our code available.
翻译:确定新闻图像的拍摄时间和地点对于记者和法证专家生成可信报道和揭露错误信息至关重要。虽然许多现有方法依赖于反向图像搜索(RIS)引擎,但这些工具常常无法返回结果,从而限制了其实际应用。在本工作中,我们解决了RIS证据不可用的挑战性场景。我们提出了NewsRECON,一种将图像与相关新闻文章关联以从文章元数据推断其日期和位置的方法。NewsRECON利用了一个包含超过90,000篇文章的语料库,并整合了:(1)用于检索事件相关文章的双编码器;(2)两个通过位置和事件一致性对文章进行重排序的交叉编码器。在TARA和5Pils-OOC数据集上的实验表明,NewsRECON优于先前的工作,并且可以与多模态大语言模型结合,在缺乏RIS证据的情况下实现新的SOTA结果。我们公开了代码。