Graph neural networks offer a promising approach to supervised learning over graph data. Graph data, especially when it is privacy-sensitive or too large to train on centrally, is often stored partitioned across disparate processing units (clients) which want to minimize the communication costs during collaborative training. The fully-distributed setup takes such partitioning to its extreme, wherein features of only a single node and its adjacent edges are kept locally with one client processor. Existing GNNs are not architected for training in such setups and incur prohibitive costs therein. We propose RETEXO, a novel transformation of existing GNNs that improves the communication efficiency during training in the fully-distributed setup. We experimentally confirm that RETEXO offers up to 6 orders of magnitude better communication efficiency even when training shallow GNNs, with a minimal trade-off in accuracy for supervised node classification tasks.

翻译：图神经网络为图数据的监督学习提供了一种有前景的方法。图数据，尤其是当涉及隐私敏感或因规模过大而无法集中训练时，通常被分区存储在不同的处理单元（客户端）上，这些客户端希望在协同训练期间最小化通信成本。完全分布式设置将这种分区推向极致，其中仅单个节点及其相邻边的特征被本地保留在一个客户端处理器上。现有的图神经网络并非为在此类设置中训练而设计，且在此场景下会产生高昂的成本。我们提出RETEXO，这是一种对现有图神经网络的创新转换，可在完全分布式设置中提高训练期间的通信效率。实验证实，即使在训练浅层图神经网络时，RETEXO也能提供高达六个数量级的通信效率提升，同时在监督节点分类任务中仅牺牲极小的准确性作为权衡。