Self-supervised learning (SSL) has gained remarkable success, for which contrastive learning (CL) plays a key role. However, the recent development of new non-CL frameworks has achieved comparable or better performance with high improvement potential, prompting researchers to enhance these frameworks further. Assimilating CL into non-CL frameworks has been thought to be beneficial, but empirical evidence indicates no visible improvements. In view of that, this paper proposes a strategy of performing CL along the dimensional direction instead of along the batch direction as done in conventional contrastive learning, named Dimensional Contrastive Learning (DimCL). DimCL aims to enhance the feature diversity, and it can serve as a regularizer to prior SSL frameworks. DimCL has been found to be effective, and the hardness-aware property is identified as a critical reason for its success. Extensive experimental results reveal that assimilating DimCL into SSL frameworks leads to performance improvement by a non-trivial margin on various datasets and backbone architectures.

翻译：自监督学习（SSL）已取得显著成功，其中对比学习（CL）发挥了关键作用。然而，近期非对比学习框架的发展以高提升潜力实现了相当或更优的性能，促使研究者进一步改进这些框架。将CL融入非对比学习框架被认为是有益的，但实证证据表明并未产生显著改进。鉴于此，本文提出一种沿维度方向而非传统对比学习中沿批次方向进行对比学习的策略，命名为维度对比学习（DimCL）。DimCL旨在增强特征多样性，可作为现有自监督学习框架的正则化器。研究发现DimCL有效，且其难度感知特性被确定为成功的关键原因。大量实验结果表明，将DimCL融入自监督学习框架可在多种数据集和骨干架构上带来显著的性能提升。