Automated deception detection is crucial for assisting humans in accurately assessing truthfulness and identifying deceptive behavior. Conventional contact-based techniques, like polygraph devices, rely on physiological signals to determine the authenticity of an individual's statements. Nevertheless, recent developments in automated deception detection have demonstrated that multimodal features derived from both audio and video modalities may outperform human observers on publicly available datasets. Despite these positive findings, the generalizability of existing audio-visual deception detection approaches across different scenarios remains largely unexplored. To close this gap, we present the first cross-domain audio-visual deception detection benchmark, that enables us to assess how well these methods generalize for use in real-world scenarios. We used widely adopted audio and visual features and different architectures for benchmarking, comparing single-to-single and multi-to-single domain generalization performance. To further exploit the impacts using data from multiple source domains for training, we investigate three types of domain sampling strategies, including domain-simultaneous, domain-alternating, and domain-by-domain for multi-to-single domain generalization evaluation. Furthermore, we proposed the Attention-Mixer fusion method to improve performance, and we believe that this new cross-domain benchmark will facilitate future research in audio-visual deception detection. Protocols and source code are available at \href{https://github.com/Redaimao/cross_domain_DD}{https://github.com/Redaimao/cross\_domain\_DD}.
翻译:自动欺骗检测对于辅助人类准确评估真实性及识别欺骗行为至关重要。传统接触式技术(如测谎仪)依赖生理信号判断个体陈述的真实性。然而,近期自动欺骗检测领域的发展表明,基于音频和视频双模态的多模态特征在公开数据集上可能超越人类观察者的表现。尽管存在这些积极发现,现有音视频欺骗检测方法在不同场景下的泛化能力仍未得到充分探索。为弥补这一不足,我们提出了首个跨领域音视频欺骗检测基准,用以评估这些方法在现实场景中的泛化性能。我们采用广泛使用的音频与视觉特征及不同架构进行基准测试,比较了单域到单域与多域到单域的泛化性能。为进一步探究多源域数据训练的影响,我们研究了三种域采样策略(域同步、域交替和逐域采样)用于多域到单域泛化评估。此外,我们提出了注意力混合器(Attention-Mixer)融合方法以提升性能,并相信这一新型跨领域基准将推动音视频欺骗检测的未来研究。相关协议与源代码已发布于\href{https://github.com/Redaimao/cross_domain_DD}{https://github.com/Redaimao/cross\_domain\_DD}。