This work analyzes Graph Neural Networks, a generalization of Fully-Connected Deep Neural Nets on Graph structured data, when their width, that is the number of nodes in each fullyconnected layer is increasing to infinity. Infinite Width Neural Networks are connecting Deep Learning to Gaussian Processes and Kernels, both Machine Learning Frameworks with long traditions and extensive theoretical foundations. Gaussian Processes and Kernels have much less hyperparameters then Neural Networks and can be used for uncertainty estimation, making them more user friendly for applications. This works extends the increasing amount of research connecting Gaussian Processes and Kernels to Neural Networks. The Kernel and Gaussian Process closed forms are derived for a variety of architectures, namely the standard Graph Neural Network, the Graph Neural Network with Skip-Concatenate Connections and the Graph Attention Neural Network. All architectures are evaluated on a variety of datasets on the task of transductive Node Regression and Classification. Additionally, a Spectral Sparsification method known as Effective Resistance is used to improve runtime and memory requirements. Extending the setting to inductive graph learning tasks (Graph Regression/ Classification) is straightforward and is briefly discussed in 3.5.
翻译:本文分析了图神经网络(一种在全连接深度神经网络基础上推广至图结构数据的模型)在其宽度(即每个全连接层节点数)趋于无穷大时的性质。无限宽神经网络将深度学习与高斯过程及核方法联系起来,这两种机器学习框架具有悠久的历史和深厚的理论基础。高斯过程与核方法的超参数远少于神经网络,且可用于不确定性估计,因此在应用中更具用户友好性。本研究拓展了当前将高斯过程与核方法联系至神经网络的日益增多的研究成果。针对多种架构推导了核函数与高斯过程的闭合形式,包括标准图神经网络、带有跳跃-拼接连接的图神经网络以及图注意力神经网络。所有架构均在多种数据集上进行了直推式节点回归与分类任务的评估。此外,采用一种称为有效电阻的谱稀疏化方法以优化运行时间与内存需求。将设置扩展至归纳式图学习任务(图回归/分类)是直接的,本文3.5节对此进行了简要讨论。