Recent works show that speech separation guided diarization (SSGD) is an increasingly promising direction, mainly thanks to the recent progress in speech separation. It performs diarization by first separating the speakers and then applying voice activity detection (VAD) on each separated stream. In this work we conduct an in-depth study of SSGD in the conversational telephone speech (CTS) domain, focusing mainly on low-latency streaming diarization applications. We consider three state-of-the-art speech separation (SSep) algorithms and study their performance both in online and offline scenarios, considering non-causal and causal implementations as well as continuous SSep (CSS) windowed inference. We compare different SSGD algorithms on two widely used CTS datasets: CALLHOME and Fisher Corpus (Part 1 and 2) and evaluate both separation and diarization performance. To improve performance, a novel, causal and computationally efficient leakage removal algorithm is proposed, which significantly decreases false alarms. We also explore, for the first time, fully end-to-end SSGD integration between SSep and VAD modules. Crucially, this enables fine-tuning on real-world data for which oracle speakers sources are not available. In particular, our best model achieves 8.8% DER on CALLHOME, which outperforms the current state-of-the-art end-to-end neural diarization model, despite being trained on an order of magnitude less data and having significantly lower latency, i.e., 0.1 vs. 1 seconds. Finally, we also show that the separated signals can be readily used also for automatic speech recognition, reaching performance close to using oracle sources in some configurations.

翻译：近期研究表明，语音分离引导的说话人日志（SSGD）正成为一个颇具前景的研究方向，这主要得益于语音分离领域的最新进展。该方法通过首先分离说话人，随后对每路分离流应用语音活动检测（VAD）来实现说话人日志。本文针对会话电话语音（CTS）领域对SSGD进行了深入研究，主要聚焦于低延迟流式说话人日志应用场景。我们选取三种最先进的语音分离（SSep）算法，分别考虑非因果与因果实现方式及连续语音分离（CSS）窗口推理，系统评估其在在线与离线场景下的性能。基于CALLHOME和Fisher语料库（第一部分与第二部分）两个广泛使用的CTS数据集，我们对比了不同SSGD算法的分离与说话人日志性能。为提升性能，提出一种新颖、因果且计算高效的泄漏消除算法，显著降低了误报率。此外，我们首次探索了SSep与VAD模块之间完全端到端的SSGD集成。这一突破性设计使得模型能够利用不含标准说话人源的真实数据进行微调。具体而言，我们的最佳模型在CALLHOME上实现了8.8%的说话人日志错误率（DER），尽管训练数据量少一个数量级且延迟显著更低（0.1秒对比1秒），仍超越了当前最先进的端到端神经说话人日志模型。最后，实验表明分离信号可有效应用于自动语音识别任务，在部分配置下其性能已接近使用标准源时的水平。