Zero-shot multi-speaker TTS aims to synthesize speech with the voice of a chosen target speaker without any fine-tuning. Prevailing methods, however, encounter limitations at adapting to new speakers of out-of-domain settings, primarily due to inadequate speaker disentanglement and content leakage. To overcome these constraints, we propose an innovative negation feature learning paradigm that models decoupled speaker attributes as deviations from the complete audio representation by utilizing the subtraction operation. By eliminating superfluous content information from the speaker representation, our negation scheme not only mitigates content leakage, thereby enhancing synthesis robustness, but also improves speaker fidelity. In addition, to facilitate the learning of diverse speaker attributes, we leverage multi-stream Transformers, which retain multiple hypotheses and instigate a training paradigm akin to ensemble learning. To unify these hypotheses and realize the final speaker representation, we employ attention pooling. Finally, in light of the imperative to generate target text utterances in the desired voice, we adopt adaptive layer normalizations to effectively fuse the previously generated speaker representation with the target text representations, as opposed to mere concatenation of the text and audio modalities. Extensive experiments and validations substantiate the efficacy of our proposed approach in preserving and harnessing speaker-specific attributes vis-`a-vis alternative baseline models.

翻译：零样本多说话人文本转语音（TTS）旨在无需任何微调，即可合成目标说话人的语音。然而，现有方法在适应域外设置中的新说话人时存在局限性，这主要归因于说话人解耦不充分和内容泄漏。为克服这些限制，我们提出一种创新的否定特征学习范式，该范式通过利用减法操作，将解耦的说话人属性建模为与完整音频表示的偏差。通过从说话人表示中消除冗余的内容信息，我们的否定方案不仅缓解了内容泄漏，从而增强了合成鲁棒性，还提高了说话人保真度。此外，为促进多样化说话人属性的学习，我们利用多流Transformer，这些Transformer保留多个假设并引发类似集成学习的训练范式。为统一这些假设并实现最终的说话人表示，我们采用注意力池化。最后，鉴于生成目标文本话语时需使用所需语音的迫切性，我们采用自适应层归一化，以有效融合先前生成的说话人表示与目标文本表示，而非简单拼接文本和音频模态。大量实验和验证证明了我们提出的方法在保留和利用说话人特定属性方面相对于其他基线模型的有效性。