In this paper, we present a Diffusion GAN based approach (Prosodic Diff-TTS) to generate the corresponding high-fidelity speech based on the style description and content text as an input to generate speech samples within only 4 denoising steps. It leverages the novel conditional prosodic layer normalization to incorporate the style embeddings into the multi head attention based phoneme encoder and mel spectrogram decoder based generator architecture to generate the speech. The style embedding is generated by fine tuning the pretrained BERT model on auxiliary tasks such as pitch, speaking speed, emotion,gender classifications. We demonstrate the efficacy of our proposed architecture on multi-speaker LibriTTS and PromptSpeech datasets, using multiple quantitative metrics that measure generated accuracy and MOS.

翻译：本文提出一种基于扩散生成对抗网络（Diffusion GAN）的方法（ProSodic Diff-TTS），以风格描述和内容文本为输入，仅需4步去噪即可生成相应的高保真语音。该方法利用新颖的条件韵律层归一化，将风格嵌入整合到基于多头注意力的音素编码器和梅尔频谱解码器生成器架构中，从而实现语音生成。风格嵌入通过微调预训练的BERT模型，在音高、语速、情感、性别分类等辅助任务上生成。我们在多说话人LibriTTS和PromptSpeech数据集上，采用多种定量指标（包括生成准确度和平均意见分数）验证了所提架构的有效性。