Graph neural networks offer a promising approach to supervised learning over graph data. Graph data, especially when it is privacy-sensitive or too large to train on centrally, is often stored partitioned across disparate processing units (clients) which want to minimize the communication costs during collaborative training. The fully-distributed setup takes such partitioning to its extreme, wherein features of only a single node and its adjacent edges are kept locally with one client processor. Existing GNNs are not architected for training in such setups and incur prohibitive costs therein. We propose RETEXO, a novel transformation of existing GNNs that improves the communication efficiency during training in the fully-distributed setup. We experimentally confirm that RETEXO offers up to 6 orders of magnitude better communication efficiency even when training shallow GNNs, with a minimal trade-off in accuracy for supervised node classification tasks.

翻译：图神经网络为图数据的监督学习提供了一种有前景的方法。图数据，特别是当它涉及隐私敏感或规模过大而无法在中心化环境下训练时，通常被分片存储在不同的处理单元（客户端）中，这些单元希望最小化协同训练期间的通信成本。全分布式设置将这种分片推向极致，其中仅单个节点及其相邻边的特征被本地保留在一个客户端处理器上。现有的图神经网络并非为在这种设置下训练而设计，并且在其中会产生高昂的成本。我们提出了RETEXO，这是一种对现有图神经网络的新颖变换，能够提高全分布式设置下训练期间的通信效率。我们通过实验证实，即使训练浅层图神经网络，RETEXO也能提供高达6个数量级的通信效率提升，同时在监督节点分类任务中仅带来极小的精度权衡。