Most existing neural-based text-to-speech methods rely on extensive datasets and face challenges under low-resource condition. In this paper, we introduce a novel semi-supervised text-to-speech synthesis model that learns from both paired and unpaired data to address this challenge. The key component of the proposed model is a dynamic quantized representation module, which is integrated into a sequential autoencoder. When given paired data, the module incorporates a trainable codebook that learns quantized representations under the supervision of the paired data. However, due to the limited paired data in low-resource scenario, these paired data are difficult to cover all phonemes. Then unpaired data is fed to expand the dynamic codebook by adding quantized representation vectors that are sufficiently distant from the existing ones during training. Experiments show that with less than 120 minutes of paired data, the proposed method outperforms existing methods in both subjective and objective metrics.

翻译：大多数现有的基于神经网络的文本到语音合成方法依赖大规模数据集，在低资源条件下面临挑战。本文提出一种新颖的半监督文本到语音合成模型，通过同时利用配对数据与非配对数据来解决该问题。该模型的核心组件是一个动态量化表示模块，该模块集成于序列自编码器中。当输入配对数据时，模块包含一个可训练的码本，在配对数据的监督下学习量化表示。然而，由于低资源场景中配对数据有限，这些数据难以覆盖所有音素。此时，非配对数据被用于扩展动态码本：在训练过程中，通过添加与现有量化表示向量距离足够远的新向量来实现扩展。实验表明，在少于120分钟配对数据的条件下，本方法在主客观指标上均优于现有方法。