Self-Supervised contrastive learning has emerged as a powerful method for obtaining high-quality representations from unlabeled data. However, feature suppression has recently been identified in standard contrastive learning ($e.g.$, SimCLR, CLIP): in a single end-to-end training stage, the contrastive model captures only parts of the shared information across contrasting views, while ignore the other potentially useful information. With feature suppression, contrastive models often fail to learn sufficient representations capable for various downstream tasks. To mitigate the feature suppression problem and ensure the contrastive model to learn comprehensive representations, we develop a novel Multistage Contrastive Learning (MCL) framework. Unlike standard contrastive learning that often result in feature suppression, MCL progressively learn new features that have not been explored in the previous stage, while maintaining the well-learned features. Extensive experiments conducted on various publicly available benchmarks validate the effectiveness of our proposed framework. In addition, we demonstrate that the proposed MCL can be adapted to a variety of popular contrastive learning backbones and boost their performance by learning features that could not be gained from standard contrastive learning procedures.

翻译：自监督对比学习已成为从无标签数据获取高质量表示的有效方法。然而，近期研究发现标准对比学习（如SimCLR、CLIP）中存在特征抑制问题：在单阶段端到端训练中，对比模型仅捕获对比视角间部分共享信息，而忽略其他潜在有用信息。由于特征抑制，对比模型往往无法学习到足以应对各类下游任务的充分表示。为缓解特征抑制问题并确保对比模型学习到全面表示，我们提出了一种新颖的多阶段对比学习（MCL）框架。与常导致特征抑制的标准对比学习不同，MCL在保持已充分学习特征的同时，逐步探索前阶段尚未习得的新特征。在多个公开基准数据集上的大量实验验证了该框架的有效性。此外，我们证明所提出的MCL可适配多种主流对比学习骨干网络，通过学习标准对比学习流程无法获取的特征来提升其性能。