Normalization is known to help the optimization of deep neural networks. Curiously, different architectures require specialized normalization methods. In this paper, we study what normalization is effective for Graph Neural Networks (GNNs). First, we adapt and evaluate the existing methods from other domains to GNNs. Faster convergence is achieved with InstanceNorm compared to BatchNorm and LayerNorm. We provide an explanation by showing that InstanceNorm serves as a preconditioner for GNNs, but such preconditioning effect is weaker with BatchNorm due to the heavy batch noise in graph datasets. Second, we show that the shift operation in InstanceNorm results in an expressiveness degradation of GNNs for highly regular graphs. We address this issue by proposing GraphNorm with a learnable shift. Empirically, GNNs with GraphNorm converge faster compared to GNNs using other normalization. GraphNorm also improves the generalization of GNNs, achieving better performance on graph classification benchmarks.

翻译：已知的正常化有助于优化深神经网络。 奇怪的是, 不同的结构需要专门的正常化方法。 在本文中, 我们研究图形神经网络( GNNs) 的正常化效果。 首先, 我们调整和评估从其它领域到 GNNs 的现有方法。 与 BatchNorm 和 Dillum Norm 相比, 更快地实现了CympleNorm 的趋同。 我们通过显示CympleNorm 是 GNNNs 的前提条件, 但是由于图表数据集中的重批量噪音, BatchNorm 的这种先决条件效果更弱。 其次, 我们显示CincentNorm 的转换导致GNNNs在高常规图形中的显性退化。 我们通过提出可学习的转变来解决这个问题。 具有Gapnorm 的GNNNMs 与 GNNs 使用其他的趋同速度更快。 Gignorm 也提高了GNNs 的通用性, 在图形分类基准上取得更好的表现。