Zero-shot multi-speaker TTS aims to synthesize speech with the voice of a chosen target speaker without any fine-tuning. Prevailing methods, however, encounter limitations at adapting to new speakers of out-of-domain settings, primarily due to inadequate speaker disentanglement and content leakage. To overcome these constraints, we propose an innovative negation feature learning paradigm that models decoupled speaker attributes as deviations from the complete audio representation by utilizing the subtraction operation. By eliminating superfluous content information from the speaker representation, our negation scheme not only mitigates content leakage, thereby enhancing synthesis robustness, but also improves speaker fidelity. In addition, to facilitate the learning of diverse speaker attributes, we leverage multi-stream Transformers, which retain multiple hypotheses and instigate a training paradigm akin to ensemble learning. To unify these hypotheses and realize the final speaker representation, we employ attention pooling. Finally, in light of the imperative to generate target text utterances in the desired voice, we adopt adaptive layer normalizations to effectively fuse the previously generated speaker representation with the target text representations, as opposed to mere concatenation of the text and audio modalities. Extensive experiments and validations substantiate the efficacy of our proposed approach in preserving and harnessing speaker-specific attributes vis-`a-vis alternative baseline models.

翻译：零样本多说话人文本到语音合成旨在无需微调即可合成具有选定目标说话人语音的语音。然而，现有方法在适应域外设置中的新说话人时存在局限性，这主要是由于说话人解耦不充分和内容泄漏所致。为克服这些限制，我们提出了一种创新的否定特征学习范式，该范式通过利用减法运算将解耦的说话人属性建模为与完整音频表示的偏差。通过从说话人表示中消除多余的内容信息，我们的否定方案不仅减轻了内容泄漏，从而增强了合成鲁棒性，还提高了说话人保真度。此外，为促进多样说话人属性的学习，我们利用了多流Transformer，它保留多个假设并引发类似集成学习的训练范式。为统一这些假设并实现最终说话人表示，我们采用了注意力池化。最后，鉴于生成以所需语音表达的目标文本的必要性，我们采用自适应层归一化来有效融合先前生成的说话人表示与目标文本表示，而非简单地拼接文本和音频模态。大量实验和验证证明了我们提出的方法在保留和利用说话人特定属性方面相较于其他基线模型的有效性。