Zero-shot text-to-speech (TTS) has gained significant attention due to its powerful voice cloning capabilities, requiring only a few seconds of unseen speaker voice prompts. However, all previous work has been developed for cloud-based systems. Taking autoregressive models as an example, although these approaches achieve high-fidelity voice cloning, they fall short in terms of inference speed, model size, and robustness. Therefore, we propose MobileSpeech, which is a fast, lightweight, and robust zero-shot text-to-speech system based on mobile devices for the first time. Specifically: 1) leveraging discrete codec, we design a parallel speech mask decoder module called SMD, which incorporates hierarchical information from the speech codec and weight mechanisms across different codec layers during the generation process. Moreover, to bridge the gap between text and speech, we introduce a high-level probabilistic mask that simulates the progression of information flow from less to more during speech generation. 2) For speaker prompts, we extract fine-grained prompt duration from the prompt speech and incorporate text, prompt speech by cross attention in SMD. We demonstrate the effectiveness of MobileSpeech on multilingual datasets at different levels, achieving state-of-the-art results in terms of generating speed and speech quality. MobileSpeech achieves RTF of 0.09 on a single A100 GPU and we have successfully deployed MobileSpeech on mobile devices. Audio samples are available at \url{https://mobilespeech.github.io/} .

翻译：零样本文本转语音（TTS）因其强大的语音克隆能力而备受关注，仅需数秒未见说话人的语音提示即可完成克隆。然而，此前所有研究均针对云端系统开发。以自回归模型为例，尽管这些方法能实现高保真语音克隆，但在推理速度、模型大小和鲁棒性方面存在不足。为此，我们首次提出MobileSpeech——一种基于移动设备的快速、轻量且鲁棒的零样本TTS系统。具体而言：1）利用离散编解码器，我们设计了名为SMD的并行语音掩码解码模块，该模块在生成过程中整合了语音编解码器的层级信息以及不同编解码层间的权重机制。此外，为弥合文本与语音之间的差距，我们引入了一种高层概率掩码，模拟语音生成过程中信息流从少到多的演变过程。2）针对说话人提示，我们从提示语音中提取细粒度提示时长，并通过SMD中的交叉注意力机制整合文本与提示语音。我们在不同级别的多语言数据集上验证了MobileSpeech的有效性，在生成速度和语音质量方面均取得了最先进的结果。在单张A100 GPU上，MobileSpeech的实时因子（RTF）达到0.09，并且我们已成功在移动设备上部署MobileSpeech。音频样本可在\url{https://mobilespeech.github.io/}获取。