Graph Neural Networks (GNN) have demonstrated state-of-the-art performance in numerous scientific and high-performance computing (HPC) applications. Recent work suggests that "souping" (combining) individually trained GNNs into a single model can improve performance without increasing compute and memory costs during inference. However, existing souping algorithms are often slow and memory-intensive, which limits their scalability. We introduce Learned Souping for GNNs, a gradient-descent-based souping strategy that substantially reduces time and memory overhead compared to existing methods. Our approach is evaluated across multiple Open Graph Benchmark (OGB) datasets and GNN architectures, achieving up to 1.2% accuracy improvement and 2.1X speedup. Additionally, we propose Partition Learned Souping, a novel partition-based variant of learned souping that significantly reduces memory usage. On the ogbn-products dataset with GraphSAGE, partition learned souping achieves a 24.5X speedup and a 76% memory reduction without compromising accuracy.

翻译：图神经网络（GNN）在众多科学计算与高性能计算（HPC）应用中展现出最先进的性能。近期研究表明，将独立训练的多个GNN模型通过"集成"方式合并为单一模型，可在不增加推理阶段计算与内存开销的前提下提升性能。然而，现有集成算法通常存在速度缓慢与内存占用过高的问题，限制了其可扩展性。本文提出基于梯度下降的GNN学习型集成策略，相比现有方法显著降低了时间与内存开销。我们在多个开放图基准（OGB）数据集和GNN架构上进行评估，该方法最高可获得1.2%的精度提升和2.1倍的加速效果。此外，我们提出分区学习型集成——一种基于分区的新型变体方法，可大幅降低内存使用。在ogbn-products数据集与GraphSAGE架构的实验中，分区学习型集成在保持精度的同时实现了24.5倍加速和76%的内存缩减。