We present a self-supervised Contrastive Video Representation Learning (CVRL) method to learn spatiotemporal visual representations from unlabeled videos. Our representations are learned using a contrastive loss, where two augmented clips from the same short video are pulled together in the embedding space, while clips from different videos are pushed away. We study what makes for good data augmentation for video self-supervised learning and find both spatial and temporal information are crucial. We carefully design data augmentations involving spatial and temporal cues. Concretely, we propose a temporally consistent spatial augmentation method to impose strong spatial augmentations on each frame of the video while maintaining the temporal consistency across frames. We also propose a sampling-based temporal augmentation method to avoid overly enforcing invariance on the clips that are distant in a video. On the Kinetics-600 dataset, a linear classifier trained on the representations learned by CVRL achieves 70.4% top-1 accuracy with a 3D-ResNet-50 (R3D-50) backbone, outperforming ImageNet supervised pre-training by 15.7% and SimCLR unsupervised pre-training by 18.8% using the same inflated R3D-50. The performance of CVRL can be further improved to 72.6% with a larger R3D-50 (4$\times$ filters) backbone, significantly closing the gap between unsupervised and supervised video representation learning.

翻译：我们展示了一种自我监督的时空视频演示学习(CVRL)方法,以从无标签的视频中学习超时视觉表现。 我们的演示是通过对比性损失学习的。 我们的演示是通过对比性损失学习的, 将同一短视频的两个强化剪辑放在嵌入空间中, 而不同视频的剪辑则被推走。 我们研究如何为视频自我监督的学习提供良好的数据增强功能, 并发现空间和时间信息都至关重要。 我们仔细设计了包含时空提示的数据增强功能。 具体地说, 我们提出了一种时间一致的空间增强方法, 以对视频的每个框架施加强大的空间增强功能, 同时保持跨框架的时间一致性。 我们还提出了一种基于取样的时空增强方法, 以避免在视频中遥远的剪辑上过度执行不易变异性。 在动能 - 600数据集中, 一个受过CVLL所学习的直线导分析器, 达到70.4%的上一级-1准确度, 3D-ResNet-50(R3D-50) 骨干, 超过图像网络前训练15.7%和SimCRR50 更大规模升级。