Singing Voice Conversion (SVC) is a technique that enables any singer to perform any song. To achieve this, it is essential to obtain speaker-agnostic representations from the source audio, which poses a significant challenge. A common solution involves utilizing a semantic-based audio pretrained model as a feature extractor. However, the degree to which the extracted features can meet the SVC requirements remains an open question. This includes their capability to accurately model melody and lyrics, the speaker-independency of their underlying acoustic information, and their robustness for in-the-wild acoustic environments. In this study, we investigate the knowledge within classical semantic-based pretrained models in much detail. We discover that the knowledge of different models is diverse and can be complementary for SVC. To jointly utilize the diverse pretrained models with mismatched time resolutions, we propose an efficient ReTrans strategy to address the feature fusion problem. Based on the above, we design a Singing Voice Conversion framework based on Diverse Semantic-based Feature Fusion (DSFF-SVC). Experimental results demonstrate that DSFF-SVC can be generalized and improve various existing SVC models, particularly in challenging real-world conversion tasks.

翻译：歌唱声音转换（SVC）是一种使任意歌手能够演唱任意歌曲的技术。为实现这一目标，从源音频中获取与说话人无关的表征至关重要，这构成了一个重大挑战。常见的解决方案是利用基于语义的音频预训练模型作为特征提取器。然而，所提取的特征能在多大程度上满足SVC的要求仍是一个开放性问题，包括其准确建模旋律与歌词的能力、底层声学信息的说话人无关性，以及对真实场景声学环境的鲁棒性。在本研究中，我们深入探究了经典基于语义的预训练模型中所蕴含的知识。我们发现不同模型的知识具有多样性，并且可以相互补充以用于SVC。为协同利用具有不匹配时间分辨率的多样化预训练模型，我们提出了一种高效的ReTrans策略以解决特征融合问题。基于上述研究，我们设计了一个基于多样化语义特征融合的歌唱声音转换框架（DSFF-SVC）。实验结果表明，DSFF-SVC具有良好的泛化能力，能够改进多种现有SVC模型，尤其在具有挑战性的真实世界转换任务中表现突出。