The goal of speech enhancement (SE) is to eliminate the background interference from the noisy speech signal. Generative models such as diffusion models (DM) have been applied to the task of SE because of better generalization in unseen noisy scenes. Technical routes for the DM-based SE methods can be summarized into three types: task-adapted diffusion process formulation, generator-plus-conditioner (GPC) structures and the multi-stage frameworks. We focus on the first two approaches, which are constructed under the GPC architecture and use the task-adapted diffusion process to better deal with the real noise. However, the performance of these SE models is limited by the following issues: (a) Non-Gaussian noise estimation in the task-adapted diffusion process. (b) Conditional domain bias caused by the weak conditioner design in the GPC structure. (c) Large amount of residual noise caused by unreasonable interpolation operations during inference. To solve the above problems, we propose a noise-aware diffusion-based SE model (NADiffuSE) to boost the SE performance, where the noise representation is extracted from the noisy speech signal and introduced as a global conditional information for estimating the non-Gaussian components. Furthermore, the anchor-based inference algorithm is employed to achieve a compromise between the speech distortion and noise residual. In order to mitigate the performance degradation caused by the conditional domain bias in the GPC framework, we investigate three model variants, all of which can be viewed as multi-stage SE based on the preprocessing networks for Mel spectrograms. Experimental results show that NADiffuSE outperforms other DM-based SE models under the GPC infrastructure. Audio samples are available at: https://square-of-w.github.io/NADiffuSE-demo/.

翻译：语音增强（SE）的目标是消除含噪语音信号中的背景干扰。生成模型（如扩散模型DM）因其在未见噪声场景中具有更好的泛化能力而被应用于SE任务。基于DM的SE方法技术路线可归纳为三类：任务自适应扩散过程构建、生成器-调节器（GPC）结构以及多阶段框架。本文聚焦于前两类方法，它们基于GPC架构构建，并采用任务自适应扩散过程以更有效地处理真实噪声。然而，这些SE模型的性能受到以下问题限制：(a) 任务自适应扩散过程中的非高斯噪声估计；(b) GPC结构中弱调节器设计导致的条件域偏差；(c) 推理过程中不合理插值操作导致的大量残余噪声。为解决上述问题，我们提出一种基于噪声感知的扩散SE模型（NADiffuSE）以提升SE性能。该模型从含噪语音信号中提取噪声表示，并将其作为全局条件信息用于估计非高斯分量。此外，采用基于锚点的推理算法实现语音失真与噪声残留之间的平衡。为缓解GPC框架中条件域偏差导致的性能退化，我们研究了三种模型变体，这些变体均可视为基于梅尔频谱预处理网络的多阶段SE。实验结果表明，NADiffuSE在GPC架构下优于其他基于DM的SE模型。音频样本可在以下链接获取：https://square-of-w.github.io/NADiffuSE-demo/。