Contrastive learning has recently established itself as a powerful self-supervised learning framework for extracting rich and versatile data representations. Broadly speaking, contrastive learning relies on a data augmentation scheme to generate two versions of the input data and learns low-dimensional representations by maximizing a normalized temperature-scaled cross entropy loss (NT-Xent) to identify augmented samples corresponding to the same original entity. In this paper, we investigate the potential of deploying contrastive learning in combination with Graph Neural Networks for embedding nodes in a graph. Specifically, we show that the quality of the resulting embeddings and training time can be significantly improved by a simple column-wise postprocessing of the embedding matrix, instead of the row-wise postprocessing via multilayer perceptrons (MLPs) that is adopted by the majority of peer methods. This modification yields improvements in downstream classification tasks of up to 1.5% and even beats existing state-of-the-art approaches on 6 out of 8 different benchmarks. We justify our choices of postprocessing by revisiting the "alignment vs. uniformity paradigm", and show that column-wise post-processing improves both "alignment" and "uniformity" of the embeddings.

翻译：对比学习最近已成为一种强大的自监督学习框架，用于提取丰富且通用的数据表示。广义上，对比学习依赖数据增强方案生成输入数据的两个版本，并通过最大化归一化温度缩放交叉熵损失（NT-Xent）来学习低维表示，以识别对应同一原始实体的增强样本。本文研究了在图神经网络中部署对比学习进行节点嵌入的潜力。具体而言，我们证明：与大多数同行方法采用的基于多层感知机（MLPs）的行方向后处理相比，通过简单的列方向后处理嵌入矩阵即可显著提升最终嵌入质量和训练时间。这一改进在下游分类任务中实现高达1.5%的性能提升，甚至在8个不同基准测试中的6个上击败现有最优方法。通过重新审视“对齐与均匀性范式”，我们验证了后处理选择的合理性，并证明列方向后处理同时改善了嵌入的“对齐性”与“均匀性”。