The over-smoothing problem is an obstacle of developing deep graph neural network (GNN). Although many approaches to improve the over-smoothing problem have been proposed, there is still a lack of comprehensive understanding and conclusion of this problem. In this work, we analyze the over-smoothing problem from the Markov chain perspective. We focus on message passing of GNN and first establish a connection between GNNs and Markov chains on the graph. GNNs are divided into two classes of operator-consistent and operator-inconsistent based on whether the corresponding Markov chains are time-homogeneous. Next we attribute the over-smoothing problem to the convergence of an arbitrary initial distribution to a stationary distribution. Based on this, we prove that although the previously proposed methods can alleviate over-smoothing, but these methods cannot avoid the over-smoothing problem. In addition, we give the conclusion of the over-smoothing problem in two types of GNNs in the Markovian sense. On the one hand, operator-consistent GNN cannot avoid over-smoothing at an exponential rate. On the other hand, operator-inconsistent GNN is not always over-smoothing. Further, we investigate the existence of the limiting distribution of the time-inhomogeneous Markov chain, from which we derive a sufficient condition for operator-inconsistent GNN to avoid over-smoothing. Finally, we design experiments to verify our findings. Results show that our proposed sufficient condition can effectively improve over-smoothing problem in operator-inconsistent GNN and enhance the performance of the model.

翻译：过平滑问题是制约深层图神经网络发展的关键障碍。尽管已有多种改善过平滑问题的方法被提出，但对该问题的系统性认知与结论仍显不足。本研究从马尔可夫链视角出发，聚焦于图神经网络的消息传递机制，首次建立了图神经网络与图上马尔可夫链之间的联系。根据对应马尔可夫链是否具有时间齐次性，将图神经网络分为算子一致型与算子非一致型两类。继而将过平滑问题归因于任意初始分布向平稳分布的收敛过程。基于此，我们证明：虽然现有方法能够缓解过平滑现象，但无法从根本上避免该问题。此外，我们从马尔可夫意义上给出了两类图神经网络中过平滑问题的结论：一方面，算子一致型图神经网络无法避免以指数速率收敛的过平滑现象；另一方面，算子非一致型图神经网络并非始终存在过平滑问题。进而，我们研究了非齐次马尔可夫链极限分布的存在性，由此推导出算子非一致型图神经网络避免过平滑的充分条件。最后，通过实验验证了理论发现，结果表明所提出的充分条件能有效改善算子非一致型图神经网络的过平滑问题并提升模型性能。