We propose a novel neural speaker diarization system using memory-aware multi-speaker embedding with sequence-to-sequence architecture (NSD-MS2S), which integrates the strengths of memory-aware multi-speaker embedding (MA-MSE) and sequence-to-sequence (Seq2Seq) architecture, leading to improvement in both efficiency and performance. Next, we further decrease the memory occupation of decoding by incorporating input features fusion and then employ a multi-head attention mechanism to capture features at different levels. NSD-MS2S achieved a macro diarization error rate (DER) of 15.9% on the CHiME-7 EVAL set, which signifies a relative improvement of 49% over the official baseline system, and is the key technique for us to achieve the best performance for the main track of CHiME-7 DASR Challenge. Additionally, we introduce a deep interactive module (DIM) in MA-MSE module to better retrieve a cleaner and more discriminative multi-speaker embedding, enabling the current model to outperform the system we used in the CHiME-7 DASR Challenge. Our code will be available at https://github.com/liyunlongaaa/NSD-MS2S.

翻译：我们提出了一种新颖的神经说话人日志系统，该系统采用基于记忆增强的多说话人嵌入与序列到序列架构（NSD-MS2S），融合了记忆增强多说话人嵌入（MA-MSE）和序列到序列（Seq2Seq）架构的优势，在效率和性能上均实现了提升。随后，我们通过引入输入特征融合进一步降低解码时的内存占用，并采用多头注意力机制捕获不同层次的特征。NSD-MS2S在CHiME-7 EVAL测试集上实现了15.9%的宏平均日志错误率（DER），相较于官方基线系统相对提升了49%，这也是我们在CHiME-7 DASR挑战赛主赛道取得最佳性能的关键技术。此外，我们在MA-MSE模块中引入了深度交互模块（DIM），以更好地获取更干净且更具判别力的多说话人嵌入，使当前模型性能优于我们在CHiME-7 DASR挑战赛中所使用的系统。我们的代码将开源在https://github.com/liyunlongaaa/NSD-MS2S。