This study introduces a refined approach to Text-to-Speech (TTS) generation that significantly enhances sampling stability across languages, with a particular focus on Hebrew. By leveraging discrete semantic units with higher phonetic correlation obtained from a self-supervised model, our method addresses the inherent instability often encountered in TTS systems, especially those dealing with non-diacriticized scripts like Hebrew. Utilizing HuBERT codes, our model generates discrete representations that are optimized for TTS tasks, thereby reducing the dependency on diacritic-based text processing. This advancement not only simplifies the language modeling process but also improves the robustness and shows controllability of the speech output due to disentenglement properties of the semantic units. The inclusion of a speaker embedding in the vocoder further aids in capturing the unique vocal characteristics of the speaker, contributing to the naturalness of the synthesized speech. Our experimental results demonstrate that this approach not only maintains high performance in Hebrew but also shows adaptability to English, underscoring its effectiveness in enhancing stability in TTS systems universally. Our method, named LOTHM (Language of The Hebrew Man), outperforms existing methods in terms of stability while achieving naturalness and speaker similarity on par with previous methods, making it a compelling choice for future speech synthesis applications. Samples can be found in our page pages.cs.huji.ac.il/adiyoss-lab/LoTHM .

翻译：本研究提出了一种改进的文本转语音生成方法，该方法显著提升了跨语言采样的稳定性，尤其侧重于希伯来语。通过利用从自监督模型中获得的具有更高语音相关性的离散语义单元，我们的方法解决了文本转语音系统中经常遇到的固有稳定性问题，特别是在处理像希伯来语这样的非变音符号文字时。利用HuBERT编码，我们的模型生成了针对文本转语音任务优化的离散表示，从而降低了对基于变音符号的文本处理的依赖。这一进展不仅简化了语言建模过程，而且由于语义单元的解纠缠特性，提高了语音输出的鲁棒性并展现了可控性。在声码器中加入说话人嵌入进一步有助于捕捉说话人独特的声学特征，从而提升合成语音的自然度。我们的实验结果表明，该方法不仅在希伯来语中保持了高性能，还显示出对英语的适应性，强调了其在普遍增强文本转语音系统稳定性方面的有效性。我们的方法命名为LOTHM（希伯来语语言模型），在稳定性方面优于现有方法，同时在自然度和说话人相似度方面与先前方法相当，使其成为未来语音合成应用的一个引人注目的选择。样本可在我们的页面pages.cs.huji.ac.il/adiyoss-lab/LoTHM找到。