The current graph neural network (GNN) systems have established a clear trend of not showing training accuracy results, and directly or indirectly relying on smaller datasets for evaluations majorly. Our in-depth analysis shows that it leads to a chain of pitfalls in the system design and evaluation process, questioning the practicality of many of the proposed system optimizations, and affecting conclusions and lessons learned. We analyze many single-GPU systems and show the fundamental impact of these pitfalls. We further develop hypotheses, recommendations, and evaluation methodologies, and provide future directions. Finally, a new reference system is developed to establish a new line of optimizations rooted in solving the system-design pitfalls efficiently and practically. The proposed design can productively be integrated into prior works, thereby truly advancing the state-of-the-art.

翻译：当前的图神经网络（GNN）系统已形成一种明显趋势：不展示训练准确度结果，且主要直接或间接依赖小型数据集进行评估。我们的深入分析表明，这会在系统设计与评估过程中引发一系列连锁陷阱，不仅质疑了诸多系统优化方案的实际可行性，更影响了研究结论与经验总结。通过对多个单GPU系统的分析，我们揭示了这些陷阱的根本影响，并进一步提出假设、建议与评估方法论，同时指明未来研究方向。最终，我们开发了一个新的基准系统，建立了基于高效且实用地解决系统设计陷阱的优化路线。该设计方案可有效地集成到先前工作中，从而真正推动该领域的前沿发展。