Real-time single-channel speech separation aims to unmix an audio stream captured from a single microphone that contains multiple people talking at once, environmental noise, and reverberation into multiple de-reverberated and noise-free speech tracks, each track containing only one talker. While large state-of-the-art DNNs can achieve excellent separation from anechoic mixtures of speech, the main challenge is to create compact and causal models that can separate reverberant mixtures at inference time. In this paper, we explore low-complexity, resource-efficient, causal DNN architectures for real-time separation of two or more simultaneous speakers. A cascade of three neural network modules are trained to sequentially perform noise-suppression, separation, and de-reverberation. For comparison, a larger end-to-end model is trained to output two anechoic speech signals directly from noisy reverberant speech mixtures. We propose an efficient single-decoder architecture with subtractive separation for real-time recursive speech separation for two or more speakers. Evaluation on real monophonic recordings of speech mixtures, according to speech separation measures like SI-SDR, perceptual measures like DNS-MOS, and a novel proposed channel separation metric, show that these compact causal models can separate speech mixtures with low latency, and perform on par with large offline state-of-the-art models like SepFormer.

翻译：实时单通道语音分离旨在将从单个麦克风捕获的、包含多人同时说话声、环境噪声和混响的音频流，分解为多个去混响、无噪声的语音轨道，每个轨道仅包含一个说话人。尽管当前最先进的大型深度神经网络能够从无混响的语音混合信号中实现出色的分离，但主要挑战在于创建紧凑且因果的模型，使其能够在推理时分离混响混合信号。本文探索了用于实时分离两个或多个同时说话人的低复杂度、资源高效且因果的深度神经网络架构。训练了一个由三个神经网络模块级联而成的系统，依次执行噪声抑制、分离和去混响操作。作为对比，还训练了一个更大的端到端模型，直接从含噪混响的混合语音中输出去混响的语音信号。我们提出了一种高效的具有减法分离机制的单解码器架构，用于两个或多个说话人的实时递归语音分离。根据语音分离指标（如SI-SDR）、感知指标（如DNS-MOS）以及新提出的通道分离指标，对真实单声道混合语音录音的评估表明，这些紧凑的因果模型能够以低延迟分离语音混合信号，其性能与离线大型先进模型（如SepFormer）相当。