Automated deception detection is crucial for assisting humans in accurately assessing truthfulness and identifying deceptive behavior. Conventional contact-based techniques, like polygraph devices, rely on physiological signals to determine the authenticity of an individual's statements. Nevertheless, recent developments in automated deception detection have demonstrated that multimodal features derived from both audio and video modalities may outperform human observers on publicly available datasets. Despite these positive findings, the generalizability of existing audio-visual deception detection approaches across different scenarios remains largely unexplored. To close this gap, we present the first cross-domain audio-visual deception detection benchmark, that enables us to assess how well these methods generalize for use in real-world scenarios. We used widely adopted audio and visual features and different architectures for benchmarking, comparing single-to-single and multi-to-single domain generalization performance. To further exploit the impacts using data from multiple source domains for training, we investigate three types of domain sampling strategies, including domain-simultaneous, domain-alternating, and domain-by-domain for multi-to-single domain generalization evaluation. We also propose an algorithm to enhance the generalization performance by maximizing the gradient inner products between modality encoders, named ``MM-IDGM". Furthermore, we proposed the Attention-Mixer fusion method to improve performance, and we believe that this new cross-domain benchmark will facilitate future research in audio-visual deception detection.

翻译：自动欺骗检测对于帮助人类准确评估真实性并识别欺骗行为至关重要。传统接触式技术（如测谎仪）依赖生理信号判断个体陈述的真实性。然而，自动欺骗检测的最新进展表明，从音频和视频模态中提取的多模态特征在公开数据集上可能超越人类观察者。尽管这些发现令人鼓舞，但现有音视频欺骗检测方法在不同场景间的泛化能力仍鲜有探索。为填补这一空白，我们提出了首个跨域音视频欺骗检测基准，用以评估这些方法在真实场景中的泛化表现。我们采用广泛使用的音频和视觉特征及不同架构进行基准测试，比较了单域到单域及多域到单域的泛化性能。为深入探究多源域训练数据的影响，我们研究了三种域采样策略（域同步、域交替及逐域采样）用于多域到单域的泛化评估。此外，我们提出了一种通过最大化模态编码器间梯度内积来增强泛化性能的算法“MM-IDGM”，并提出了Attention-Mixer融合方法以提升性能。我们相信这一新型跨域基准将推动音视频欺骗检测领域的未来研究。