Expressive speech synthesis is crucial for many human-computer interaction scenarios, such as audiobooks, podcasts, and voice assistants. Previous works focus on predicting the style embeddings at one single scale from the information within the current sentence. Whereas, context information in neighboring sentences and multi-scale nature of style in human speech are neglected, making it challenging to convert multi-sentence text into natural and expressive speech. In this paper, we propose MSStyleTTS, a style modeling method for expressive speech synthesis, to capture and predict styles at different levels from a wider range of context rather than a sentence. Two sub-modules, including multi-scale style extractor and multi-scale style predictor, are trained together with a FastSpeech 2 based acoustic model. The predictor is designed to explore the hierarchical context information by considering structural relationships in context and predict style embeddings at global-level, sentence-level and subword-level. The extractor extracts multi-scale style embedding from the ground-truth speech and explicitly guides the style prediction. Evaluations on both in-domain and out-of-domain audiobook datasets demonstrate that the proposed method significantly outperforms the three baselines. In addition, we conduct the analysis of the context information and multi-scale style representations that have never been discussed before.

翻译：富有表现力的语音合成对于许多人机交互场景至关重要，如有声书、播客和语音助手。以往的研究主要关注从当前句子内部信息中预测单一尺度的风格嵌入。然而，相邻句子中的上下文信息以及人类语音中风格的多尺度特性被忽略，这使得将多句子文本转换为自然且富有表现力的语音变得具有挑战性。本文提出MSStyleTTS，一种用于富有表现力语音合成的风格建模方法，旨在从更广泛的上下文（而非单个句子）中捕捉和预测不同层次的风格。两个子模块——多尺度风格提取器和多尺度风格预测器——与基于FastSpeech 2的声学模型共同训练。预测器通过考虑上下文的层次结构关系来探索层次化上下文信息，并在全局级、句子级和子词级预测风格嵌入。提取器从真实语音中提取多尺度风格嵌入，并显式指导风格预测。对域内和域外有声书数据集的评估表明，所提方法显著优于三种基线方法。此外，本文还对此前尚未探讨过的上下文信息和多尺度风格表示进行了分析。