In this paper, we propose a new Self-Supervised Learning (SSL) algorithm called data2vec-aqc, for speech representation learning from unlabeled speech data. Our goal is to improve SSL for speech in domains where both unlabeled and labeled data are limited. Building on the recently introduced data2vec, we introduce additional modules to the data2vec framework that leverage the benefit of data augmentations, quantized representations, and clustering. The interaction between these modules helps solve the cross-contrastive loss as an additional self-supervised objective. data2vec-aqc achieves up to 14.1% and 20.9% relative WER improvement over the existing state-of-the-art data2vec system over the test-clean and test-other sets, respectively of LibriSpeech, without the use of any language model (LM). Our proposed model also achieves up to 17.8\% relative WER gains over the baseline data2vec when fine-tuned on a subset of the Switchboard dataset. Code: https://github.com/Speech-Lab-IITM/data2vec-aqc.

翻译：本文提出了一种名为data2vec-aqc的新型自监督学习（SSL）算法，用于从无标签语音数据中学习语音表征。我们的目标是在无标签数据和标签数据均有限的领域中改进语音的SSL。基于近期提出的data2vec框架，我们引入额外的模块，利用数据增强、量化表征和聚类的优势。这些模块之间的交互有助于将交叉对比损失作为额外的自监督目标进行求解。在不使用任何语言模型（LM）的情况下，data2vec-aqc在LibriSpeech的test-clean和test-other测试集上，相较于现有最先进的data2vec系统，分别实现了高达14.1%和20.9%的相对词错误率（WER）降低。当在Switchboard数据集子集上进行微调时，我们提出的模型相较于基线data2vec模型还实现了高达17.8%的相对WER降低。代码：https://github.com/Speech-Lab-IITM/data2vec-aqc。