Link prediction is a crucial graph-learning task with applications including citation prediction and product recommendation. Distilling Graph Neural Networks (GNNs) teachers into Multi-Layer Perceptrons (MLPs) students has emerged as an effective approach to achieve strong performance and reducing computational cost by removing graph dependency. However, existing distillation methods only use standard GNNs and overlook alternative teachers such as specialized model for link prediction (GNN4LP) and heuristic methods (e.g., common neighbors). This paper first explores the impact of different teachers in GNN-to-MLP distillation. Surprisingly, we find that stronger teachers do not always produce stronger students: MLPs distilled from GNN4LP can underperform those distilled from simpler GNNs, while weaker heuristic methods can teach MLPs to near-GNN performance with drastically reduced training costs. Building on these insights, we propose Ensemble Heuristic-Distilled MLPs (EHDM), which eliminates graph dependencies while effectively integrating complementary signals via a gating mechanism. Experiments on ten datasets show an average 7.93% improvement over previous GNN-to-MLP approaches with 1.95-3.32 times less training time, indicating EHDM is an efficient and effective link prediction method.

翻译：链接预测是图学习中的关键任务，其应用包括引文预测和产品推荐。将图神经网络教师模型蒸馏为多层感知器学生模型已成为一种有效方法，通过消除图依赖性来实现强大性能并降低计算成本。然而，现有蒸馏方法仅使用标准图神经网络，忽略了链接预测专用模型和启发式方法等替代教师模型。本文首次探究了不同教师模型在图神经网络到多层感知器蒸馏中的影响。令人惊讶的是，我们发现更强的教师并不总能培养出更强的学生：从链接预测专用模型蒸馏得到的多层感知器可能表现不如从更简单图神经网络蒸馏的模型，而较弱的启发式方法却能够以大幅降低的训练成本将多层感知器训练至接近图神经网络的性能。基于这些发现，我们提出了集成启发式蒸馏多层感知器，该方法在消除图依赖性的同时，通过门控机制有效整合互补信号。在十个数据集上的实验表明，该方法较先前图神经网络到多层感知器方法平均提升7.93%性能，同时训练时间减少1.95-3.32倍，证明集成启发式蒸馏多层感知器是一种高效且有效的链接预测方法。