Face manipulation technology is advancing very rapidly, and new methods are being proposed day by day. The aim of this work is to propose a deepfake detector that can cope with the wide variety of manipulation methods and scenarios encountered in the real world. Our key insight is that each person has specific characteristics that a synthetic generator likely cannot reproduce. Accordingly, we extract audio-visual features which characterize the identity of a person, and use them to create a person-of-interest (POI) deepfake detector. We leverage a contrastive learning paradigm to learn the moving-face and audio segment embeddings that are most discriminative for each identity. As a result, when the video and/or audio of a person is manipulated, its representation in the embedding space becomes inconsistent with the real identity, allowing reliable detection. Training is carried out exclusively on real talking-face video; thus, the detector does not depend on any specific manipulation method and yields the highest generalization ability. In addition, our method can detect both single-modality (audio-only, video-only) and multi-modality (audio-video) attacks, and is robust to low-quality or corrupted videos. Experiments on a wide variety of datasets confirm that our method ensures a SOTA performance, especially on low quality videos. Code is publicly available on-line at https://github.com/grip-unina/poi-forensics.

翻译：面部操控技术发展极为迅速，新型方法层出不穷。本研究旨在提出一种能够应对现实世界中多种操控方法与场景的深度伪造检测器。我们的关键洞察在于：每个人具有合成生成器难以复现的独特特征。据此，我们提取表征人物身份的音视频特征，构建特定人物（POI）深度伪造检测器。通过对比学习范式，我们学习每个身份最具区分度的动态人脸与音频片段嵌入表示。当人物的视频和/或音频被篡改时，其在嵌入空间中的表征将与真实身份产生不一致，从而实现可靠检测。模型仅使用真实说话人脸视频进行训练，因此不依赖任何特定操控方法，具有最高泛化能力。此外，本方法可同时检测单模态（仅音频/仅视频）与多模态（音视频）攻击，并对低质量或受损视频具有鲁棒性。在多种数据集上的实验证实，本方法尤其在低质量视频上实现了最先进的性能。代码已开源：https://github.com/grip-unina/poi-forensics