Self-supervised learning (SSL) approaches have achieved great success when the amount of labeled data is limited. Within SSL, models learn robust feature representations by solving pretext tasks. One such pretext task is contrastive learning, which involves forming pairs of similar and dissimilar input samples, guiding the model to distinguish between them. In this work, we investigate the application of contrastive learning to the domain of medical image analysis. Our findings reveal that MoCo v2, a state-of-the-art contrastive learning method, encounters dimensional collapse when applied to medical images. This is attributed to the high degree of inter-image similarity shared between the medical images. To address this, we propose two key contributions: local feature learning and feature decorrelation. Local feature learning improves the ability of the model to focus on the local regions of the image, while feature decorrelation removes the linear dependence among the features. Our experimental findings demonstrate that our contributions significantly enhance the model's performance in the downstream task of medical segmentation, both in the linear evaluation and full fine-tuning settings. This work illustrates the importance of effectively adapting SSL techniques to the characteristics of medical imaging tasks.

翻译：自监督学习方法在标记数据有限的情况下已取得巨大成功。在自监督学习中，模型通过解决前置任务来学习鲁棒的特征表示。其中一种前置任务是对比学习，它涉及构建相似与不相似输入样本对，引导模型区分它们。本研究探讨了对比学习在医学图像分析领域的应用。我们发现，最先进的对比学习方法MoCo v2在应用于医学图像时会遭遇维度坍塌问题。这是由于医学图像之间存在高度的图像间相似性。为解决此问题，我们提出两项关键贡献：局部特征学习与特征去相关。局部特征学习提升了模型关注图像局部区域的能力，而特征去相关则消除了特征间的线性依赖关系。实验结果表明，在医学分割的下游任务中，无论是线性评估还是完全微调设置下，我们的贡献均显著提升了模型性能。本研究阐明了根据医学成像任务特性有效调整自监督学习方法的重要性。