The standard approach to contrastive learning is to maximize the agreement between different views of the data. The views are ordered in pairs, such that they are either positive, encoding different views of the same object, or negative, corresponding to views of different objects. The supervisory signal comes from maximizing the total similarity over positive pairs, while the negative pairs are needed to avoid collapse. In this work, we note that the approach of considering individual pairs cannot account for both intra-set and inter-set similarities when the sets are formed from the views of the data. It thus limits the information content of the supervisory signal available to train representations. We propose to go beyond contrasting individual pairs of objects by focusing on contrasting objects as sets. For this, we use combinatorial quadratic assignment theory designed to evaluate set and graph similarities and derive set-contrastive objective as a regularizer for contrastive learning methods. We conduct experiments and demonstrate that our method improves learned representations for the tasks of metric learning and self-supervised classification.

翻译：对比学习的标准方法是最大化数据不同视图之间的一致性。这些视图按配对排序，其中正对编码同一对象的不同视图，负对则对应不同对象的视图。监督信号来自最大化正对的总相似性，而负对用于避免模型坍塌。本文指出，当视图构成集合时，仅考虑单个配对的方法无法同时捕捉集合内和集合间的相似性，从而限制了可用于训练表示的监督信号的信息量。我们提出超越对比单个对象对，转而聚焦于对象集合之间的对比。为此，我们利用专为评估集合与图相似性而设计的组合二次分配理论，推导出集合对比目标函数作为对比学习方法的正则化项。实验表明，我们的方法在度量学习和自监督分类任务中改进了学习到的表示。