This paper proposes a method to construct pretext tasks for self-supervised learning on group equivariant neural networks. Group equivariant neural networks are the models whose structure is restricted to commute with the transformations on the input. Therefore, it is important to construct pretext tasks for self-supervised learning that do not contradict this equivariance. To ensure that training is consistent with the equivariance, we propose two concepts for self-supervised tasks: equivariant pretext labels and invariant contrastive loss. Equivariant pretext labels use a set of labels on which we can define the transformations that correspond to the input change. Invariant contrastive loss uses a modified contrastive loss that absorbs the effect of transformations on each input. Experiments on standard image recognition benchmarks demonstrate that the equivariant neural networks exploit the proposed equivariant self-supervised tasks.

翻译：本文提出了一种在群等变神经网络上构建自监督学习预文任务的方法。群等变神经网络是结构受限于与输入变换可交换的模型。因此，构建不与这种等变性矛盾的自监督学习预文任务至关重要。为确保训练与等变性一致，我们提出了两种自监督任务概念：等变预文标签和不变对比损失。等变预文标签利用一组标签，可定义与输入变化对应的变换。不变对比损失则采用改进的对比损失，吸收每个输入上变换的影响。标准图像识别基准实验表明，等变神经网络能够有效利用所提出的等变自监督任务。