Online Speech Enhancement was mainly reserved for predictive models. A key advantage of these models is that for an incoming signal frame from a stream of data, the model is called only once for enhancement. In contrast, generative Speech Enhancement models often require multiple calls, resulting in a computational complexity that is too high for many online speech enhancement applications. This work presents the Diffusion Buffer, a generative diffusion-based Speech Enhancement model which only requires one neural network call per incoming signal frame from a stream of data and performs enhancement in an online fashion on a consumer-grade GPU. The key idea of the Diffusion Buffer is to align physical time with Diffusion time-steps. The approach progressively denoises frames through physical time, where past frames have more noise removed. Consequently, an enhanced frame is output to the listener with a delay defined by the Diffusion Buffer, and the output frame has a corresponding look-ahead. In this work, we extend upon our previous work by carefully designing a 2D convolutional UNet architecture that specifically aligns with the Diffusion Buffer's look-ahead. We observe that the proposed UNet improves performance, particularly when the algorithmic latency is low. Moreover, we show that using a Data Prediction loss instead of Denoising Score Matching loss enables flexible control over the trade-off between algorithmic latency and quality during inference. The extended Diffusion Buffer equipped with a novel NN and loss function drastically reduces the algorithmic latency from 320 - 960 ms to 32 - 176 ms with an even increased performance. While it has been shown before that offline generative diffusion models outperform predictive approaches in unseen noisy speech data, we confirm that the online Diffusion Buffer also outperforms its predictive counterpart on unseen noisy speech data.

翻译：在线语音增强主要局限于预测模型。这些模型的一个关键优势在于，对于数据流中的输入信号帧，模型仅需调用一次即可完成增强。相比之下，生成式语音增强模型通常需要多次调用，导致计算复杂度过高，难以适用于多数在线语音增强场景。本研究提出扩散缓冲区——一种基于扩散过程的生成式语音增强模型，该模型对数据流中每帧输入信号仅需进行一次神经网络调用，并能在消费级GPU上以在线方式执行增强。扩散缓冲区的核心思想是将物理时间与扩散时间步对齐。该方法通过物理时间逐步对信号帧进行去噪，使得历史帧获得更充分的噪声消除。因此，增强后的音频帧会以扩散缓冲区定义的延迟输出给听者，且输出帧具有相应的前瞻量。本研究在先前工作基础上，精心设计了与扩散缓冲区前瞻机制精准匹配的二维卷积UNet架构。实验表明，所提出的UNet结构能有效提升性能，尤其在算法延迟较低时效果更为显著。此外，我们证明采用数据预测损失函数替代去噪分数匹配损失，可在推理过程中灵活权衡算法延迟与增强质量。配备新型神经网络与损失函数的扩展版扩散缓冲区，将算法延迟从320-960毫秒大幅降低至32-176毫秒，同时性能得到进一步提升。尽管已有研究证明离线生成式扩散模型在未知噪声语音数据上优于预测方法，我们在此验证了在线扩散缓冲区同样能在未知噪声语音数据上超越其对应的预测模型。