Oversmoothing is a common phenomenon observed in graph neural networks (GNNs), in which an increase in the network depth leads to a deterioration in their performance. Graph contrastive learning (GCL) is emerging as a promising way of leveraging vast unlabeled graph data. As a marriage between GNNs and contrastive learning, it remains unclear whether GCL inherits the same oversmoothing defect from GNNs. This work undertakes a fundamental analysis of GCL from the perspective of oversmoothing on the first hand. We demonstrate empirically that increasing network depth in GCL also leads to oversmoothing in their deep representations, and surprisingly, the shallow ones. We refer to this phenomenon in GCL as `long-range starvation', wherein lower layers in deep networks suffer from degradation due to the lack of sufficient guidance from supervision. Based on our findings, we present BlockGCL, a remarkably simple yet effective blockwise training framework to prevent GCL from notorious oversmoothing. Without bells and whistles, BlockGCL consistently improves robustness and stability for well-established GCL methods with increasing numbers of layers on several real-world graph benchmarks.

翻译：过度平滑是图神经网络中常见的一种现象，即网络深度增加会导致其性能下降。图对比学习作为利用大量无标注图数据的一种新兴方法，结合了图神经网络与对比学习，但目前尚不清楚图对比学习是否继承了图神经网络的过度平滑缺陷。本文首次从过度平滑的角度对图对比学习进行了基础性分析。我们通过实验证明，在图对比学习中增加网络深度不仅会导致深层表示出现过度平滑，甚至浅层表示也会如此。我们将图对比学习中的这一现象称为“长程饥饿”，即深度网络中较低层因缺乏来自监督的充分引导而性能退化。基于我们的发现，我们提出了BlockGCL——一种极其简单而有效的分块训练框架，用于防止图对比学习出现恼人的过度平滑问题。无需额外复杂设计，BlockGCL在多个真实世界图基准数据集上，能够随着层数增加持续提升现有图对比学习方法的鲁棒性和稳定性。