The recent renaissance in generative models, driven primarily by the advent of diffusion models and iterative improvement in GAN methods, has enabled many creative applications. However, each advancement is also accompanied by a rise in the potential for misuse. In the arena of deepfake generation this is a key societal issue. In particular, the ability to modify segments of videos using such generative techniques creates a new paradigm of deepfakes which are mostly real videos altered slightly to distort the truth. Current deepfake detection methods in the academic literature are not evaluated on this paradigm. In this paper, we present a deepfake detection method able to address this issue by performing both frame and video level deepfake prediction. To facilitate testing our method we create a new benchmark dataset where videos have both real and fake frame sequences. Our method utilizes the Vision Transformer, Scaling and Shifting pretraining and Timeseries Transformer to temporally segment videos to help facilitate the interpretation of possible deepfakes. Extensive experiments on a variety of deepfake generation methods show excellent results on temporal segmentation and classical video level predictions as well. In particular, the paradigm we introduce will form a powerful tool for the moderation of deepfakes, where human oversight can be better targeted to the parts of videos suspected of being deepfakes. All experiments can be reproduced at: https://github.com/sanjaysaha1311/temporal-deepfake-segmentation.

翻译：近年来，生成模型的复兴——主要由扩散模型的出现和GAN方法的迭代改进推动——催生了诸多创造性应用。然而，每项进步也伴随着滥用风险的增加。在深度伪造生成领域，这已成为关键的社会议题。特别是，利用此类生成技术修改视频片段的能力，创造了一种新的深度伪造范式：视频主体内容真实，仅通过局部篡改扭曲事实。现有学术文献中的深度伪造检测方法尚未基于此范式进行评估。本文提出一种能够通过帧级与视频级双重预测解决该问题的深度伪造检测方法。为便于测试，我们构建了一个包含真实与伪造帧序列混合视频的基准数据集。该方法采用Vision Transformer、Scaling and Shifting预训练技术及Timeseries Transformer对视频进行时间分割，以辅助深度伪造片段的判读。在多种深度伪造生成方法上的大量实验表明，该方法在时间分割与经典视频级预测上均取得优异效果。特别地，本文提出的范式将成为深度伪造审核的有力工具，使人工监督能更精准地聚焦于疑似深度伪造的视频片段。所有实验均可通过 https://github.com/sanjaysaha1311/temporal-deepfake-segmentation 复现。