While song generation and singing voice conversion (SVC) have evolved significantly, they have long been developed isolated: the former lacks zero-shot speaker cloning, while the latter overlooks vocal-accompaniment synergy. To bridge this gap, we propose UniSinger, the first end-to-end framework unifying speaker cloning song generation and accompaniment co-generation SVC. Building on the multimodal diffusion transformer, we construct a unified speaker embedding space transferring speaker representation from SVC to song generation, endowing fine-grained cross-task timbre control. To mitigate multi-task optimization conflicts, we design a curriculum learning strategy using task-specific modality masking to guide the model to gradually master the generative mechanisms among semantic content, vocal timbre, and accompaniment. Experiments show state-of-the-art performance on both tasks and realizes complementary benefits, offering new possibilities for intelligent music production.

翻译：尽管歌曲生成与歌声转换（SVC）领域已取得显著进展，但长期以来两者独立发展：前者缺乏零样本说话人克隆能力，后者则忽视人声与伴奏的协同性。为弥合这一鸿沟，我们提出UniSinger——首个统一说话人克隆歌曲生成与伴奏协同生成歌声转换的端到端框架。基于多模态扩散Transformer，我们构建统一说话人嵌入空间，将SVC中的说话人表征迁移至歌曲生成，实现跨任务细粒度音色控制。为缓解多任务优化冲突，我们设计课程学习策略，通过任务特异性模态掩码引导模型逐步掌握语义内容、人声音色与伴奏之间的生成机制。实验表明，本方法在两项任务中均取得最先进性能，并实现互补增益，为智能音乐制作开辟新可能。