Predicting turn-taking in multiparty conversations has many practical applications in human-computer/robot interaction. However, the complexity of human communication makes it a challenging task. Recent advances have shown that synchronous multi-perspective egocentric data can significantly improve turn-taking prediction compared to asynchronous, single-perspective transcriptions. Building on this research, we propose a new multimodal transformer-based architecture for predicting turn-taking in embodied, synchronized multi-perspective data. Our experimental results on the recently introduced EgoCom dataset show a substantial performance improvement of up to 14.01% on average compared to existing baselines and alternative transformer-based approaches. The source code, and the pre-trained models of our 3M-Transformer will be available upon acceptance.

翻译：预测多人对话中的话轮转换在人机交互/机器人交互中具有多种实际应用。然而，人类交流的复杂性使其成为一项具有挑战性的任务。最新研究表明，与异步、单视角转录相比，同步多视角自我中心数据能显著提升话轮转换预测性能。基于这一研究，我们提出了一种新的多模态Transformer架构，用于预测具身式同步多视角数据中的话轮转换。我们在近期发布的EgoCom数据集上的实验结果显示，与现有基线方法及其他基于Transformer的方法相比，平均性能提升了高达14.01%。我们的3M-Transformer的源代码及预训练模型将在论文被接收后公开提供。