Graph Contrastive Learning (GCL) aims to learn node representations by aligning positive pairs and separating negative ones. However, limited research has been conducted on the inner law behind specific augmentations used in graph-based learning. What kind of augmentation will help downstream performance, how does contrastive learning actually influence downstream tasks, and why the magnitude of augmentation matters? This paper seeks to address these questions by establishing a connection between augmentation and downstream performance, as well as by investigating the generalization of contrastive learning. Our findings reveal that GCL contributes to downstream tasks mainly by separating different classes rather than gathering nodes of the same class. So perfect alignment and augmentation overlap which draw all intra-class samples the same can not explain the success of contrastive learning. Then in order to comprehend how augmentation aids the contrastive learning process, we conduct further investigations into its generalization, finding that perfect alignment that draw positive pair the same could help contrastive loss but is poisonous to generalization, on the contrary, imperfect alignment enhances the model's generalization ability. We analyse the result by information theory and graph spectrum theory respectively, and propose two simple but effective methods to verify the theories. The two methods could be easily applied to various GCL algorithms and extensive experiments are conducted to prove its effectiveness.
翻译:图对比学习旨在通过对齐正样本对并分离负样本来学习节点表示。然而,目前关于图学习中特定增广操作背后内在规律的研究仍十分有限。何种增广有助于下游任务性能?对比学习如何实际影响下游任务?增广的幅度为何重要?本文旨在通过建立增广与下游任务表现之间的联系,并探究对比学习的泛化能力来回答这些问题。我们的发现表明,图对比学习主要通过分离不同类别而非聚集同类节点来促进下游任务。因此,将同一类别内所有样本绘制成相同表示的完美对齐与增广重叠无法解释对比学习的成功。为了理解增广如何辅助对比学习过程,我们进一步研究了其泛化性,发现使正样本对完全相同的完美对齐虽有助于降低对比损失,却对泛化有害;相反,非完美对齐能够增强模型的泛化能力。我们分别从信息论和图谱理论角度分析这一结果,并提出两种简单有效的方法来验证这些理论。这两种方法可轻松应用于各类图对比学习算法,并通过大量实验证明了其有效性。