Recently, zero-shot TTS and VC methods have gained attention due to their practicality of being able to generate voices even unseen during training. Among these methods, zero-shot modifications of the VITS model have shown superior performance, while having useful properties inherited from VITS. However, the performance of VITS and VITS-based zero-shot models vary dramatically depending on how the losses are balanced. This can be problematic, as it requires a burdensome procedure of tuning loss balance hyper-parameters to find the optimal balance. In this work, we propose a novel framework that finds this optimum without search, by inducing the decoder of VITS-based models to its full reconstruction ability. With our framework, we show superior performance compared to baselines in zero-shot TTS and VC, achieving state-of-the-art performance. Furthermore, we show the robustness of our framework in various settings. We provide an explanation for the results in the discussion.

翻译：近年来，零样本文本转语音（TTS）和语音转换（VC）方法因其能够在训练中生成未见语音的实用性而受到关注。在这些方法中，基于VITS模型的零样本变体展现了优越性能，同时继承了VITS的有用特性。然而，VITS及其零样本模型的性能会因损失平衡方式的不同而发生显著变化。这可能导致问题，因为需要繁琐的损失平衡超参数调优过程以找到最优平衡。在本工作中，我们提出了一种无需搜索即可自动找到最优平衡的新型框架，通过引导基于VITS模型的解码器实现其完全重构能力。采用我们的框架，在零样本TTS和VC任务中相比基线方法展示了更优性能，实现了最先进水平。此外，我们验证了该框架在多种设置下的鲁棒性。本文讨论部分对结果进行了详细解释。