We introduce OpenVoice, a versatile voice cloning approach that requires only a short audio clip from the reference speaker to replicate their voice and generate speech in multiple languages. OpenVoice represents a significant advancement in addressing the following open challenges in the field: 1) Flexible Voice Style Control. OpenVoice enables granular control over voice styles, including emotion, accent, rhythm, pauses, and intonation, in addition to replicating the tone color of the reference speaker. The voice styles are not directly copied from and constrained by the style of the reference speaker. Previous approaches lacked the ability to flexibly manipulate voice styles after cloning. 2) Zero-Shot Cross-Lingual Voice Cloning. OpenVoice achieves zero-shot cross-lingual voice cloning for languages not included in the massive-speaker training set. Unlike previous approaches, which typically require extensive massive-speaker multi-lingual (MSML) dataset for all languages, OpenVoice can clone voices into a new language without any massive-speaker training data for that language. OpenVoice is also computationally efficient, costing tens of times less than commercially available APIs that offer even inferior performance. To foster further research in the field, we have made the source code and trained model publicly accessible. We also provide qualitative results in our demo website. Prior to its public release, our internal version of OpenVoice was used tens of millions of times by users worldwide between May and October 2023, serving as the backend of MyShell.

翻译：我们提出OpenVoice，一种多才多艺的语音克隆方法，仅需参考说话者的一段简短音频片段即可复制其声音并生成多种语言的语音。OpenVoice在解决该领域的以下公开挑战方面取得了重大进展：1）灵活的声音风格控制。OpenVoice能够在复制参考说话者音色的同时，实现声音风格的精细控制，包括情感、口音、节奏、停顿和语调。声音风格并非直接复制自或受限于参考说话者的风格。先前的方法在克隆后缺乏灵活操控声音风格的能力。2）零样本跨语言语音克隆。OpenVoice实现了对未包含在大规模说话者训练集中的语言的零样本跨语言语音克隆。与先前方法（通常需要为所有语言准备广泛的大规模说话者多语言（MSML）数据集）不同，OpenVoice可以将语音克隆到一种新语言中，而无需该语言的大规模说话者训练数据。OpenVoice还具有计算效率高的特点，其成本比性能甚至更差的商用API低数十倍。为了促进该领域的进一步研究，我们已公开提供源代码和训练模型。我们还在演示网站上提供了定性结果。在公开发布之前，我们的内部版本OpenVoice在2023年5月至10月期间已由全球用户使用了数千万次，并作为MyShell的后端服务。