Previous research has shown that established techniques for spoken voice conversion (VC) do not perform as well when applied to singing voice conversion (SVC). We propose an alternative loss component in a loss function that is otherwise well-established among VC tasks, which has been shown to improve our model's SVC performance. We first trained a singer identity embedding (SIE) network on mel-spectrograms of singer recordings to produce singer-specific variance encodings using contrastive learning. We subsequently trained a well-known autoencoder framework (AutoVC) conditioned on these SIEs, and measured differences in SVC performance when using different latent regressor loss components. We found that using this loss w.r.t. SIEs leads to better performance than w.r.t. bottleneck embeddings, where converted audio is more natural and specific towards target singers. The inclusion of this loss component has the advantage of explicitly forcing the network to reconstruct with timbral similarity, and also negates the effect of poor disentanglement in AutoVC's bottleneck embeddings. We demonstrate peculiar diversity between computational and human evaluations on singer-converted audio clips, which highlights the necessity of both. We also propose a pitch-matching mechanism between source and target singers to ensure these evaluations are not influenced by differences in pitch register.

翻译：先前研究表明，在语音转换（VC）中成熟的技术应用于歌声转换（SVC）时性能欠佳。我们针对VC任务中已被广泛验证的损失函数提出了一种替代损失分量，该分量能有效提升模型在SVC任务中的表现。我们首先基于歌手录音的梅尔频谱图训练了歌手身份嵌入网络（SIE），通过对比学习生成歌手特定的变异编码。随后，我们以这些SIE为条件训练了经典的自动编码器框架（AutoVC），并测量了使用不同潜在回归器损失分量时SVC性能的差异。实验发现，相对于瓶颈嵌入，基于SIE的损失能显著提升转换效果——转换后的音频更加自然且更贴近目标歌手的音色特质。该损失分量的引入具有双重优势：既强制网络以音色相似性为目标进行重构，又抵消了AutoVC瓶颈嵌入中解耦不充分的影响。我们还揭示了计算评估与人工评估在歌手转换音频片段上的显著差异，这凸显了两种评估方式的互补必要性。此外，我们提出了源歌手与目标歌手之间的音高匹配机制，以确保评估结果不受音域差异的干扰。