Contrastive Learning has recently achieved state-of-the-art performance in a wide range of tasks. Many contrastive learning approaches use mined hard negatives to make batches more informative during training but these approaches are inefficient as they increase epoch length proportional to the number of mined negatives and require frequent updates of nearest neighbor indices or mining from recent batches. In this work, we provide an alternative to hard negative mining, Global Contrastive Batch Sampling (GCBS), an efficient approximation to the batch assignment problem that upper bounds the gap between the global and training losses, $\mathcal{L}^{Global} - \mathcal{L}^{Train}$, in contrastive learning settings. Through experimentation we find GCBS improves state-of-the-art performance in sentence embedding and code-search tasks. Additionally, GCBS is easy to implement as it requires only a few additional lines of code, does not maintain external data structures such as nearest neighbor indices, is more computationally efficient than the most minimal hard negative mining approaches, and makes no changes to the model being trained.

翻译：对比学习近期在广泛任务中取得了最先进的性能。许多对比学习方法利用挖掘的难负样本使训练过程中的批次更具信息性，但这些方法效率低下，因为其会按挖掘的负样本数量比例增加训练周期长度，且需要频繁更新最近邻索引或从最近批次中挖掘数据。本文提出了一种替代难负样本挖掘的方法——全局对比批采样（GCBS），这是一种对批次分配问题的高效近似方法，可在对比学习设置中上界全局损失与训练损失之间的差距，即$\mathcal{L}^{Global} - \mathcal{L}^{Train}$。实验表明，GCBS在句子嵌入和代码搜索任务中提升了最先进的性能。此外，GCBS易于实现，仅需额外几行代码，无需维护最近邻索引等外部数据结构，计算上比最简化的难负样本挖掘方法更为高效，且不改变正在训练的模型结构。