Humans use context to assess the veracity of information. However, current audio deepfake detectors only analyze the audio file without considering either context or transcripts. We create and analyze a Journalist-provided Deepfake Dataset (JDD) of 255 public deepfakes which were primarily contributed by over 70 journalists since early 2024. We also generate a synthetic audio dataset (SYN) of dead public figures and propose a novel Context-based Audio Deepfake Detector (CADD) architecture. In addition, we evaluate performance on two large-scale datasets: ITW and P$^2$V. We show that sufficient context and/or the transcript can significantly improve the efficacy of audio deepfake detectors. Performance (measured via F1 score, AUC, and EER) of multiple baseline audio deepfake detectors and traditional classifiers can be improved by 5%-37.58% in F1-score, 3.77%-42.79% in AUC, and 6.17%-47.83% in EER. We additionally show that CADD, via its use of context and/or transcripts, is more robust to 5 adversarial evasion strategies, limiting performance degradation to an average of just -0.71% across all experiments. Code, models, and datasets are available at our project page: https://sites.northwestern.edu/nsail/cadd-context-based-audio-deepfake-detection (access restricted during review).

翻译：人类依赖上下文信息来评估信息的真实性。然而，当前的音频深度伪造检测器仅分析音频文件本身，未考虑上下文或文本转录。我们创建并分析了由记者提供的深度伪造数据集（JDD），该数据集包含255个公众人物的深度伪造音频，主要由2024年初以来超过70名记者贡献。我们还生成了一个已故公众人物的合成音频数据集（SYN），并提出了一种新颖的基于上下文的音频深度伪造检测器（CADD）架构。此外，我们在两个大规模数据集（ITW和P$^2$V）上评估了性能。研究表明，充分的上下文信息和/或文本转录能显著提升音频深度伪造检测器的效能。多个基线音频深度伪造检测器和传统分类器的性能（通过F1分数、AUC和EER衡量）可获得显著提升：F1分数提高5%-37.58%，AUC提高3.77%-42.79%，EER改善6.17%-47.83%。我们还证明，CADD通过利用上下文和/或文本转录，对5种对抗性规避策略具有更强的鲁棒性，在所有实验中平均性能衰减仅为-0.71%。代码、模型和数据集可在项目页面获取：https://sites.northwestern.edu/nsail/cadd-context-based-audio-deepfake-detection（审稿期间访问受限）。