The performance of acoustic models degrades notably in noisy environments. Speech enhancement (SE) can be used as a front-end strategy to aid automatic speech recognition (ASR) systems. However, existing training objectives of SE methods are not fully effective at integrating speech-text and noisy-clean paired data for training toward unseen ASR systems. In this study, we propose a general denoising framework, D4AM, for various downstream acoustic models. Our framework fine-tunes the SE model with the backward gradient according to a specific acoustic model and the corresponding classification objective. In addition, our method aims to consider the regression objective as an auxiliary loss to make the SE model generalize to other unseen acoustic models. To jointly train an SE unit with regression and classification objectives, D4AM uses an adjustment scheme to directly estimate suitable weighting coefficients rather than undergoing a grid search process with additional training costs. The adjustment scheme consists of two parts: gradient calibration and regression objective weighting. The experimental results show that D4AM can consistently and effectively provide improvements to various unseen acoustic models and outperforms other combination setups. Specifically, when evaluated on the Google ASR API with real noisy data completely unseen during SE training, D4AM achieves a relative WER reduction of 24.65% compared with the direct feeding of noisy input. To our knowledge, this is the first work that deploys an effective combination scheme of regression (denoising) and classification (ASR) objectives to derive a general pre-processor applicable to various unseen ASR systems. Our code is available at https://github.com/ChangLee0903/D4AM.

翻译：声学模型在噪声环境中的性能显著下降。语音增强（SE）可作为前端策略辅助自动语音识别（ASR）系统。然而，现有SE方法的训练目标未能充分利用语音-文本和含噪-干净配对数据以实现对未见ASR系统的训练。本研究提出通用降噪框架D4AM，适用于各类下游声学模型。该框架根据特定声学模型及其对应的分类目标，通过反向梯度对SE模型进行微调。此外，本方法将回归目标作为辅助损失函数，使SE模型能泛化至其他未见声学模型。为联合训练兼具回归与分类目标的SE单元，D4AM采用一种调整策略直接估计合适的权重系数，避免需额外训练成本的网格搜索过程。该调整策略包含两部分：梯度校准与回归目标加权。实验结果表明，D4AM能持续有效地提升各类未见声学模型性能，并优于其他组合设置。具体而言，当以SE训练中完全未见真实噪声数据在Google ASR API上进行评估时，与直接输入含噪信号相比，D4AM实现相对词错误率（WER）降低24.65%。据我们所知，这是首个通过有效组合回归（降噪）与分类（ASR）目标以构建适用于各类未见ASR系统的通用预处理器的研究工作。代码开源于https://github.com/ChangLee0903/D4AM。