Graph convolutional networks (GCNs) have been proved to be very practical to handle various graph-related tasks. It has attracted considerable research interest to study deep GCNs, due to their potential superior performance compared with shallow ones. However, simply increasing network depth will, on the contrary, hurt the performance due to the over-smoothing problem. Adding residual connection is proved to be effective for learning deep convolutional neural networks (deep CNNs), it is not trivial when applied to deep GCNs. Recent works proposed an initial residual mechanism that did alleviate the over-smoothing problem in deep GCNs. However, according to our study, their algorithms are quite sensitive to different datasets. In their setting, the personalization (dynamic) and correlation (evolving) of how residual applies are ignored. To this end, we propose a novel model called Dynamic evolving initial Residual Graph Convolutional Network (DRGCN). Firstly, we use a dynamic block for each node to adaptively fetch information from the initial representation. Secondly, we use an evolving block to model the residual evolving pattern between layers. Our experimental results show that our model effectively relieves the problem of over-smoothing in deep GCNs and outperforms the state-of-the-art (SOTA) methods on various benchmark datasets. Moreover, we develop a mini-batch version of DRGCN which can be applied to large-scale data. Coupling with several fair training techniques, our model reaches new SOTA results on the large-scale ogbn-arxiv dataset of Open Graph Benchmark (OGB). Our reproducible code is available on GitHub.

翻译：图卷积网络已被证明在处理各类图相关任务中非常实用。由于深度图卷积网络相比浅层网络具有潜在更优性能，研究深度图卷积网络已引起广泛关注。然而，单纯增加网络深度反而会因过平滑问题损害性能。尽管添加残差连接被证明对学习深度卷积神经网络有效，但将其应用于深度图卷积网络时并非易事。近期研究提出的初始残差机制确实缓解了深度图卷积网络中的过平滑问题。然而，根据我们的研究，这些算法对不同数据集相当敏感。在其设置中，残差应用的个性化（动态性）和相关性（演化性）被忽视。为此，我们提出名为动态演化初始残差图卷积网络（DRGCN）的新模型。首先，我们对每个节点使用动态块，以自适应地从初始表示中获取信息。其次，我们使用演化块来建模层间残差演化模式。实验结果表明，我们的模型有效缓解了深度图卷积网络中的过平滑问题，并在多个基准数据集上超越现有最优方法。此外，我们开发了DRGCN的小批量版本，可适用于大规模数据。结合若干公平训练技术，我们的模型在Open Graph Benchmark（OGB）的大规模ogbn-arxiv数据集上达到了新的最优结果。我们的可复现代码已在GitHub上公开。