In this paper, we introduce a zero-shot Voice Transfer (VT) module that can be seamlessly integrated into a multi-lingual Text-to-speech (TTS) system to transfer an individual's voice across languages. Our proposed VT module comprises a speaker-encoder that processes reference speech, a bottleneck layer, and residual adapters, connected to preexisting TTS layers. We compare the performance of various configurations of these components and report Mean Opinion Score (MOS) and Speaker Similarity across languages. Using a single English reference speech per speaker, we achieve an average voice transfer similarity score of 73% across nine target languages. Vocal characteristics contribute significantly to the construction and perception of individual identity. The loss of one's voice, due to physical or neurological conditions, can lead to a profound sense of loss, impacting one's core identity. As a case study, we demonstrate that our approach can not only transfer typical speech but also restore the voices of individuals with dysarthria, even when only atypical speech samples are available - a valuable utility for those who have never had typical speech or banked their voice. Cross-lingual typical audio samples, plus videos demonstrating voice restoration for dysarthric speakers are available here (google.github.io/tacotron/publications/zero_shot_voice_transfer).

翻译：本文提出一种零样本语音迁移模块，可无缝集成到多语言文本转语音系统中，实现个体语音的跨语言转换。该语音迁移模块包含处理参考语音的说话人编码器、瓶颈层及残差适配器，并与现有TTS层相连接。我们比较了这些组件的不同配置性能，并报告了跨语言的平均意见得分和说话人相似度。使用每位说话人的单段英语参考语音，我们在九种目标语言中实现了平均73%的语音迁移相似度得分。声音特征对个体身份的构建与感知具有重要影响。因生理或神经疾病导致的声音丧失会引发强烈的失落感，进而影响个体的核心身份认同。作为案例研究，我们证明该方法不仅能迁移典型语音，还能为构音障碍患者恢复声音——即使仅能获得非典型语音样本，这对于从未拥有典型语音或未进行语音存档的群体具有重要应用价值。跨语言典型音频样本及构音障碍患者语音恢复演示视频可通过此链接获取（google.github.io/tacotron/publications/zero_shot_voice_transfer）。