In recent years, self-supervised contrastive learning has emerged as a distinguished paradigm in the artificial intelligence landscape. It facilitates unsupervised feature learning through contrastive delineations at the instance level. However, crafting an effective self-supervised paradigm remains a pivotal challenge within this field. This paper delves into two crucial factors impacting self-supervised contrastive learning-bach size and pretext tasks, and from a data processing standpoint, proposes an adaptive technique of batch fusion. The proposed method, via dimensionality reduction and reconstruction of batch data, enables formerly isolated individual data to partake in intra-batch communication through the Embedding Layer. Moreover, it adaptively amplifies the self-supervised feature encoding capability as the training progresses. We conducted a linear classification test of this method based on the classic contrastive learning framework on ImageNet-1k. The empirical findings illustrate that our approach achieves state-of-the-art performance under equitable comparisons. Benefiting from its "plug-and-play" characteristics, we further explored other contrastive learning methods. On the ImageNet-100, compared to the original performance, the top1 has seen a maximum increase of 1.25%. We suggest that the proposed method may contribute to the advancement of data-driven self-supervised learning research, bringing a fresh perspective to this community.
翻译:近年来,自监督对比学习已成为人工智能领域中一种显著范式,它通过实例级别的对比划分促进无监督特征学习。然而,如何设计有效的自监督范式仍是该领域的关键挑战。本文深入探讨了影响自监督对比学习的两个关键因素——批量大小与预文本任务,并从数据处理角度提出了一种批量融合的自适应技术。该方法通过对批量数据进行降维与重构,使得原本孤立的个体数据能够通过嵌入层参与批量内通信。此外,随着训练的进行,该方法能自适应地增强自监督特征编码能力。我们在ImageNet-1k上基于经典对比学习框架对该方法进行了线性分类测试。实验结果表明,在公平比较下,我们的方法达到了最先进的性能。得益于其"即插即用"特性,我们进一步探索了其他对比学习方法。在ImageNet-100上,与原始性能相比,top1精度最大提升了1.25%。我们建议,所提方法可能有助于推动数据驱动的自监督学习研究,为该领域带来新的视角。