Gaussian processes (GPs) are an attractive class of machine learning models because of their simplicity and flexibility as building blocks of more complex Bayesian models. Meanwhile, graph neural networks (GNNs) emerged recently as a promising class of models for graph-structured data in semi-supervised learning and beyond. Their competitive performance is often attributed to a proper capturing of the graph inductive bias. In this work, we introduce this inductive bias into GPs to improve their predictive performance for graph-structured data. We show that a prominent example of GNNs, the graph convolutional network, is equivalent to some GP when its layers are infinitely wide; and we analyze the kernel universality and the limiting behavior in depth. We further present a programmable procedure to compose covariance kernels inspired by this equivalence and derive example kernels corresponding to several interesting members of the GNN family. We also propose a computationally efficient approximation of the covariance matrix for scalable posterior inference with large-scale data. We demonstrate that these graph-based kernels lead to competitive classification and regression performance, as well as advantages in computation time, compared with the respective GNNs.

翻译：高斯过程因其作为更复杂贝叶斯模型构建块时的简洁性和灵活性，而成为一类极具吸引力的机器学习模型。与此同时，图神经网络近年来在半监督学习及更广泛的图结构数据处理领域崭露头角，成为一类有前景的模型。其优越性能通常归因于对图结构归纳偏置的有效捕获。在本工作中，我们将这种归纳偏置引入高斯过程，以提升其对图结构数据的预测性能。我们证明图神经网络的典型代表——图卷积网络，在其层宽趋于无穷时等价于某个高斯过程；并对核的通用性和极限行为进行了深入分析。进一步地，我们提出一种可编程的协方差核构建流程（基于该等价关系），并推导出对应于图神经网络家族中若干重要成员的示例核函数。此外，我们提出一种协方差矩阵的计算高效近似方法，以支持大规模数据下的可扩展后验推断。实验表明，与相应的图神经网络相比，这些基于图的核函数在分类与回归任务中取得了具有竞争力的性能，并在计算时间上展现出优势。