In recent years, graph neural networks (GNNs) have achieved great success in the field of graph representation learning. Although prior work has shed light into the expressiveness of those models (\ie whether they can distinguish pairs of non-isomorphic graphs), it is still not clear what structural information is encoded into the node representations that are learned by those models. In this paper, we investigate which properties of graphs are captured purely by these models, when no node attributes are available. Specifically, we study four popular GNN models, and we show that two of them embed all nodes into the same feature vector, while the other two models generate representations that are related to the number of walks over the input graph. Strikingly, structurally dissimilar nodes can have similar representations at some layer $k>1$, if they have the same number of walks of length $k$. We empirically verify our theoretical findings on real datasets.

翻译：近年来，图神经网络（GNNs）在图表示学习领域取得了巨大成功。尽管已有研究揭示了这些模型的表达能力（即它们能否区分非同构的图对），但模型学习到的节点表示中编码了哪些结构信息仍不清楚。本文研究了在没有节点特征的情况下，这些模型仅凭自身能捕获图的哪些属性。具体而言，我们分析了四种主流的GNN模型，发现其中两种模型将所有节点嵌入到相同的特征向量中，而另外两种模型生成的表示与输入图上的游走次数相关。值得注意的是，当结构不相似的节点具有相同长度的游走次数时，它们在某些层$k>1$上可能表现出相似的表示。我们在真实数据集上通过实验验证了这些理论发现。