Stutter removal is an essential scenario in the field of speech editing. However, when the speech recording contains stutters, the existing text-based speech editing approaches still suffer from: 1) the over-smoothing problem in the edited speech; 2) lack of robustness due to the noise introduced by stutter; 3) to remove the stutters, users are required to determine the edited region manually. To tackle the challenges in stutter removal, we propose FluentSpeech, a stutter-oriented automatic speech editing model. Specifically, 1) we propose a context-aware diffusion model that iteratively refines the modified mel-spectrogram with the guidance of context features; 2) we introduce a stutter predictor module to inject the stutter information into the hidden sequence; 3) we also propose a stutter-oriented automatic speech editing (SASE) dataset that contains spontaneous speech recordings with time-aligned stutter labels to train the automatic stutter localization model. Experimental results on VCTK and LibriTTS datasets demonstrate that our model achieves state-of-the-art performance on speech editing. Further experiments on our SASE dataset show that FluentSpeech can effectively improve the fluency of stuttering speech in terms of objective and subjective metrics. Code and audio samples can be found at https://github.com/Zain-Jiang/Speech-Editing-Toolkit.

翻译：口吃消除是语音编辑领域中的一个重要应用场景。然而，当语音录音包含口吃现象时，现有基于文本的语音编辑方法仍面临以下挑战：1) 编辑后的语音存在过度平滑问题；2) 由于口吃引入的噪声导致鲁棒性不足；3) 用户需要手动确定编辑区域才能消除口吃。为应对口吃消除中的这些挑战，我们提出了FluentSpeech——一种面向口吃的自动语音编辑模型。具体地：1) 我们提出了一种上下文感知扩散模型，该模型在上下文特征引导下对修正后的梅尔频谱图进行迭代精炼；2) 引入口吃预测模块，将口吃信息注入隐含序列；3) 构建了面向口吃的自动语音编辑（SASE）数据集，包含带时间对齐口吃标注的自发语音录音，用于训练自动口吃定位模型。在VCTK和LibriTTS数据集上的实验结果表明，我们的模型在语音编辑任务上达到了最优性能。进一步在SASE数据集上的实验证明，FluentSpeech在客观与主观指标上均能有效改善口吃语音的流畅度。代码与音频样本请访问 https://github.com/Zain-Jiang/Speech-Editing-Toolkit。