What matters for contrastive learning? We argue that contrastive learning heavily relies on informative features, or "hard" (positive or negative) features. Early works include more informative features by applying complex data augmentations and large batch size or memory bank, and recent works design elaborate sampling approaches to explore informative features. The key challenge toward exploring such features is that the source multi-view data is generated by applying random data augmentations, making it infeasible to always add useful information in the augmented data. Consequently, the informativeness of features learned from such augmented data is limited. In response, we propose to directly augment the features in latent space, thereby learning discriminative representations without a large amount of input data. We perform a meta learning technique to build the augmentation generator that updates its network parameters by considering the performance of the encoder. However, insufficient input data may lead the encoder to learn collapsed features and therefore malfunction the augmentation generator. A new margin-injected regularization is further added in the objective function to avoid the encoder learning a degenerate mapping. To contrast all features in one gradient back-propagation step, we adopt the proposed optimization-driven unified contrastive loss instead of the conventional contrastive loss. Empirically, our method achieves state-of-the-art results on several benchmark datasets.

翻译：摘要：对比学习的关键是什么？我们认为对比学习高度依赖于信息丰富的特征，即“困难”（正向或负向）特征。早期工作通过复杂的数增强、大批量或记忆库来引入更多信息特征，近期工作则设计精妙的采样策略以探索信息特征。探索此类特征的核心挑战在于：源多视图数据通过随机数据增强生成，导致增强数据无法始终包含有用信息，因此从这类增强数据中学习到的特征信息量有限。为此，我们提出直接在潜在空间中增强特征，从而无需大量输入数据即可学习判别性表示。我们采用元学习技术构建增强生成器，通过考虑编码器的性能来更新其网络参数。然而，输入数据不足可能导致编码器学习到坍缩特征，进而使增强生成器失效。我们在目标函数中进一步引入新型边缘注入正则化项，以避免编码器学习退化的映射。为实现单次梯度反向传播中对比所有特征，我们采用所提出的优化驱动统一对比损失替代传统对比损失。实验表明，本方法在多个基准数据集上取得了最先进的性能。