Despite rapid advancement in recent years, current speech enhancement models often produce speech that differs in perceptual quality from real clean speech. We propose a learning objective that formalizes differences in perceptual quality, by using domain knowledge of acoustic-phonetics. We identify temporal acoustic parameters -- such as spectral tilt, spectral flux, shimmer, etc. -- that are non-differentiable, and we develop a neural network estimator that can accurately predict their time-series values across an utterance. We also model phoneme-specific weights for each feature, as the acoustic parameters are known to show different behavior in different phonemes. We can add this criterion as an auxiliary loss to any model that produces speech, to optimize speech outputs to match the values of clean speech in these features. Experimentally we show that it improves speech enhancement workflows in both time-domain and time-frequency domain, as measured by standard evaluation metrics. We also provide an analysis of phoneme-dependent improvement on acoustic parameters, demonstrating the additional interpretability that our method provides. This analysis can suggest which features are currently the bottleneck for improvement.

翻译：尽管近年来语音增强模型取得了快速进展，但当前模型生成的语音在感知质量上仍常与真实纯净语音存在差异。我们提出了一种利用声学-语音学领域知识形式化感知质量差异的学习目标。我们识别出诸如频谱倾斜、频谱通量、微光等非可微的时间域声学参数，并开发了一种神经网络估计器，能够准确预测这些参数在整段语音中的时序值。我们还为每个特征建模了音素级权重，因为不同音素中声学参数的表现存在已知差异。该准则可作为辅助损失函数添加到任何生成语音的模型中，通过优化语音输出使其在这些特征参数上与纯净语音的值匹配。实验表明，无论是在时域还是时频域，该方法均能通过标准评估指标改善语音增强流程。我们还提供了基于音素的声学参数改进分析，展示了本方法带来的额外可解释性，这种分析有助于识别当前制约性能提升的瓶颈特征。