We propose WaveTrainerFit, a neural vocoder that performs high-quality waveform generation from data-driven features such as SSL features. WaveTrainerFit builds upon the WaveFit vocoder, which integrates diffusion model and generative adversarial network. Furthermore, the proposed method incorporates the following key improvements: 1. By introducing trainable priors, the inference process starts from noise close to the target speech instead of Gaussian noise. 2. Reference-aware gain adjustment is performed by imposing constraints on the trainable prior to matching the speech energy. These improvements are expected to reduce the complexity of waveform modeling from data-driven features, enabling high-quality waveform generation with fewer inference steps. Through experiments, we showed that WaveTrainerFit can generate highly natural waveforms with improved speaker similarity from data-driven features, while requiring fewer iterations than WaveFit. Moreover, we showed that the proposed method works robustly with respect to the depth at which SSL features are extracted. Code and pre-trained models are available from https://github.com/line/WaveTrainerFit.

翻译：我们提出了WaveTrainerFit，这是一种能够从数据驱动特征（例如SSL特征）生成高质量波形的神经声码器。WaveTrainerFit建立在WaveFit声码器的基础上，后者融合了扩散模型和生成对抗网络。此外，所提出的方法引入了以下关键改进：1. 通过引入可训练先验，推理过程从接近目标语音的噪声开始，而非高斯噪声。2. 通过对可训练先验施加约束以匹配语音能量，进行参考感知的增益调整。这些改进有望降低从数据驱动特征进行波形建模的复杂度，从而以更少的推理步骤实现高质量的波形生成。通过实验，我们证明了WaveTrainerFit能够从数据驱动特征生成高度自然且说话人相似度更高的波形，同时所需的迭代次数少于WaveFit。此外，我们还证明了所提出的方法对于SSL特征提取的深度具有鲁棒性。代码和预训练模型可从 https://github.com/line/WaveTrainerFit 获取。