Creating realistic and natural-sounding synthetic speech remains a big challenge for voice identities unseen during training. As there is growing interest in synthesizing voices of new speakers, here we investigate the ability of normalizing flows in text-to-speech (TTS) and voice conversion (VC) modes to extrapolate from speakers observed during training to create unseen speaker identities. Firstly, we create an approach for TTS and VC, and then we comprehensively evaluate our methods and baselines in terms of intelligibility, naturalness, speaker similarity, and ability to create new voices. We use both objective and subjective metrics to benchmark our techniques on 2 evaluation tasks: zero-shot and new voice speech synthesis. The goal of the former task is to measure the precision of the conversion to an unseen voice. The goal of the latter is to measure the ability to create new voices. Extensive evaluations demonstrate that the proposed approach systematically allows to obtain state-of-the-art performance in zero-shot speech synthesis and creates various new voices, unobserved in the training set. We consider this work to be the first attempt to synthesize new voices based on mel-spectrograms and normalizing flows, along with a comprehensive analysis and comparison of the TTS and VC modes.

翻译：生成逼真且自然的合成语音对于训练中未见过的语音身份仍是一个重大挑战。随着合成新说话者语音的需求日益增长，本文研究了归一化流在文本转语音（TTS）和语音转换（VC）模式下，从训练中观察到的说话者进行外推以生成未见语音身份的能力。首先，我们提出了一种适用于TTS和VC的方法，随后在可理解性、自然度、说话者相似度以及创造新语音的能力方面，对我们的方法和基线进行了全面评估。我们采用客观与主观两种指标，在两个评估任务上对技术进行基准测试：零样本语音合成和新语音合成。前者的目标是衡量向未见语音转换的精确度，后者的目标是衡量生成新语音的能力。大量评估表明，所提方法能够系统地获得零样本语音合成的最新性能，并生成训练集中未出现的多种新语音。我们认为，这项工作是首次尝试基于梅尔频谱图和归一化流合成新语音，并对TTS与VC模式进行了全面分析与比较。