Pause insertion, also known as phrase break prediction and phrasing, is an essential part of TTS systems because proper pauses with natural duration significantly enhance the rhythm and intelligibility of synthetic speech. However, conventional phrasing models ignore various speakers' different styles of inserting silent pauses, which can degrade the performance of the model trained on a multi-speaker speech corpus. To this end, we propose more powerful pause insertion frameworks based on a pre-trained language model. Our approach uses bidirectional encoder representations from transformers (BERT) pre-trained on a large-scale text corpus, injecting speaker embedding to capture various speaker characteristics. We also leverage duration-aware pause insertion for more natural multi-speaker TTS. We develop and evaluate two types of models. The first improves conventional phrasing models on the position prediction of respiratory pauses (RPs), i.e., silent pauses at word transitions without punctuation. It performs speaker-conditioned RP prediction considering contextual information and is used to demonstrate the effect of speaker information on the prediction. The second model is further designed for phoneme-based TTS models and performs duration-aware pause insertion, predicting both RPs and punctuation-indicated pauses (PIPs) that are categorized by duration. The evaluation results show that our models improve the precision and recall of pause insertion and the rhythm of synthetic speech.

翻译：停顿插入（亦称短语边界预测或分句）是语音合成系统的重要组成部分，因为自然时长的合理停顿能显著增强合成语音的韵律和清晰度。然而传统分句模型忽视了不同说话人插入静默停顿的差异化风格，这可能导致基于多说话人语音语料库训练的模型性能下降。为此，我们提出基于预训练语言模型的更强大停顿插入框架。该方法采用在大规模文本语料上预训练的BERT（Transformer双向编码器表示），通过注入说话人嵌入来捕捉不同说话人特征，并利用时长感知的停顿插入实现更自然的多说话人语音合成。我们开发并评估了两种模型：第一种模型改进了传统分句模型对呼吸停顿位置（即无标点符号的词间无声停顿）的预测能力，通过考虑上下文信息实现说话人条件化的呼吸停顿预测，用以验证说话人信息对预测的影响；第二种模型专为基于音素的语音合成系统设计，通过时长感知的停顿插入，同时预测呼吸停顿和按时长分类的标点提示停顿。评估结果表明，我们的模型提升了停顿插入的精确率与召回率，并改善了合成语音的韵律表现。