This work aims to improve unsupervised audio-visual pre-training. Inspired by the efficacy of data augmentation in visual contrastive learning, we propose a novel speed co-augmentation method that randomly changes the playback speeds of both audio and video data. Despite its simplicity, the speed co-augmentation method possesses two compelling attributes: (1) it increases the diversity of audio-visual pairs and doubles the size of negative pairs, resulting in a significant enhancement in the learned representations, and (2) it changes the strict correlation between audio-visual pairs but introduces a partial relationship between the augmented pairs, which is modeled by our proposed SoftInfoNCE loss to further boost the performance. Experimental results show that the proposed method significantly improves the learned representations when compared to vanilla audio-visual contrastive learning.

翻译：本文旨在改进无监督视听预训练。受数据增强在视觉对比学习中的有效性启发，我们提出了一种新颖的速度协同增强方法，该方法可随机改变音频与视频数据的播放速度。尽管方法简单，速度协同增强具有两个显著特性：(1) 它增加了视听对的多样性并将负样本对数量翻倍，从而显著增强学习到的表征；(2) 它改变了视听对之间的严格相关性，但在增强对之间引入了部分关联性，通过我们提出的SoftInfoNCE损失函数进行建模，以进一步提升性能。实验结果表明，与原始视听对比学习相比，所提方法显著提升了学习到的表征质量。