To extract robust deep representations from long sequential modeling of speech data, we propose a self-supervised learning approach, namely Contrastive Separative Coding (CSC). Our key finding is to learn such representations by separating the target signal from contrastive interfering signals. First, a multi-task separative encoder is built to extract shared separable and discriminative embedding; secondly, we propose a powerful cross-attention mechanism performed over speaker representations across various interfering conditions, allowing the model to focus on and globally aggregate the most critical information to answer the "query" (current bottom-up embedding) while paying less attention to interfering, noisy, or irrelevant parts; lastly, we form a new probabilistic contrastive loss which estimates and maximizes the mutual information between the representations and the global speaker vector. While most prior unsupervised methods have focused on predicting the future, neighboring, or missing samples, we take a different perspective of predicting the interfered samples. Moreover, our contrastive separative loss is free from negative sampling. The experiment demonstrates that our approach can learn useful representations achieving a strong speaker verification performance in adverse conditions.

翻译：为了从长长的顺序模拟语音数据中获取强有力的深度表述,我们建议采取自我监督的学习方法,即相互分离编码(CSC)。我们的关键发现是通过将目标信号与对比干扰信号分开来学习这种表述。首先,建立一个多任务分离编码器,以提取共享的分离和歧视性嵌入;其次,我们建议一种强大的跨语音代表器,在不同干扰条件下对发言者代表器实施强有力的交叉注意机制,使该模式能够集中并在全球范围内汇总最关键的信息,以回答“尖叫”(目前的自下而上嵌入),同时较少注意干扰、吵闹或无关的部分;最后,我们形成了一种新的概率对比损失,用以估计和最大限度地增加陈述与全球扬声器矢量之间的相互信息。虽然大多数以前未受监督的方法都侧重于预测未来、邻接或缺失的样本,但我们从不同的角度预测被干扰的样本。此外,我们的对比性分离损失没有负面抽样。实验表明,我们的方法可以学到有用的表述方式,在不利条件下实现强烈的发言者性表现。