Self-supervised learning, or SSL, holds the key to expanding the usage of machine learning in real-world tasks by alleviating heavy human supervision. Contrastive learning and its varieties have been SSL strategies in various fields. We use margins as a stepping stone for understanding how contrastive learning works at a deeper level and providing potential directions to improve representation learning. Through gradient analysis, we found that margins scale gradients in three different ways: emphasizing positive samples, de-emphasizing positive samples when angles of positive samples are wide, and attenuating the diminishing gradients as the estimated probability approaches the target probability. We separately analyze each and provide possible directions for improving SSL frameworks. Our experimental results demonstrate that these properties can contribute to acquiring better representations, which can enhance performance in both seen and unseen datasets.

翻译：自监督学习（SSL）通过减轻对大量人工标注的依赖，成为推动机器学习在现实任务中广泛应用的关键。对比学习及其变体已成为多个领域中自监督学习的主要策略。我们以边界为切入点，深入理解对比学习的工作机制，并为改进表示学习提供潜在方向。通过梯度分析，我们发现边界以三种方式对梯度进行缩放：强化正样本、在正样本角度较大时减弱其强调作用，以及当估计概率趋近目标概率时抑制梯度衰减。我们对每种机制进行了单独分析，并提出了改进自监督学习框架的可能方向。实验结果表明，这些特性有助于获取更优的表示，从而在可见及未见数据集上均能提升性能。