Deception detection is gaining increasing interest due to ethical and security concerns. This paper explores the application of convolutional neural networks for the purpose of multimodal deception detection. We use a dataset built by interviewing 104 subjects about two topics, with one truthful and one falsified response from each subject about each topic. In particular, we make three main contributions. First, we extract linguistic and physiological features from this data to train and construct the neural network models. Second, we propose a fused convolutional neural network model using both modalities in order to achieve an improved overall performance. Third, we compare our new approach with earlier methods designed for multimodal deception detection. We find that our system outperforms regular classification methods; our results indicate the feasibility of using neural networks for deception detection even in the presence of limited amounts of data.

翻译：欺骗检测因伦理与安全关切而日益受到关注。本文探索卷积神经网络在多模态欺骗检测中的应用。我们使用通过访谈104名受试者构建的数据集，涵盖两个话题，每位受试者对每个话题分别提供真实与虚假回答。具体而言，本研究作出三项主要贡献：首先，我们从数据中提取言语与生理特征以训练并构建神经网络模型；其次，我们提出一种融合双模态的卷积神经网络模型，以提升整体性能；第三，我们将新方法与早期多模态欺骗检测方法进行比较。实验表明，本系统优于常规分类方法；研究结果证实了即使在数据量有限的情况下，使用神经网络进行欺骗检测仍具有可行性。