Studies continually find that message-passing graph convolutional networks suffer from the over-smoothing issue. Basically, the issue of over-smoothing refers to the phenomenon that the learned embeddings for all nodes can become very similar to one another and therefore are uninformative after repeatedly applying message passing iterations. Intuitively, we can expect the generated embeddings become smooth asymptotically layerwisely, that is each layer of graph convolution generates a smoothed version of embeddings as compared to that generated by the previous layer. Based on this intuition, we propose RandAlign, a stochastic regularization method for graph convolutional networks. The idea of RandAlign is to randomly align the learned embedding for each node with that of the previous layer using randomly interpolation in each graph convolution layer. Through alignment, the smoothness of the generated embeddings is explicitly reduced. To better maintain the benefit yielded by the graph convolution, in the alignment step we introduce to first scale the embedding of the previous layer to the same norm as the generated embedding and then perform random interpolation for aligning the generated embedding. RandAlign is a parameter-free method and can be directly applied without introducing additional trainable weights or hyper-parameters. We experimentally evaluate RandAlign on different graph domain tasks on seven benchmark datasets. The experimental results show that RandAlign is a general method that improves the generalization performance of various graph convolutional network models and also improves the numerical stability of optimization, advancing the state of the art performance for graph representation learning.

翻译：研究不断发现消息传递图卷积网络存在过度平滑问题。本质上，过度平滑是指所有节点学习到的嵌入表示在经过多次消息传递迭代后彼此高度相似、从而丧失信息性的现象。直观而言，我们可以预期生成的嵌入表示会逐层渐进地趋于平滑，即每一层图卷积生成的嵌入表示相较于上一层都会产生更平滑的版本。基于这一直觉，我们提出RandAlign，一种针对图卷积网络的随机正则化方法。RandAlign的核心思想是在每个图卷积层中，通过随机插值将每个节点当前学习到的嵌入表示与上一层的嵌入表示进行随机对齐。通过这种对齐操作，嵌入表示的平滑度被显著降低。为更好地保留图卷积带来的收益，我们在对齐步骤中引入缩放操作：先将上一层嵌入表示缩放至与当前层生成嵌入表示相同的范数，然后再执行随机插值以对齐当前层嵌入表示。RandAlign是一种无参数方法，可直接应用而无需引入额外的可训练权重或超参数。我们在七个基准数据集上对不同图域任务进行了实验评估。实验结果表明，RandAlign是一种通用方法，既能提升各种图卷积网络模型的泛化性能，又能改善优化的数值稳定性，从而推动了图表示学习领域的最新性能水平。