Graph Neural Networks (GNNs) are a pertinent tool for any machine learning task due to their ability to learn functions over graph structures, a powerful and expressive data representation. The detection of communities, an unsupervised task has increasingly been performed with GNNs. Clustering nodes in a graph using the multi-dimensionality of node features with the connectivity of the graph has many applications to real world tasks from social networks to genomics. Unfortunately, there is currently a gap in the literature with no established sufficient benchmarking environment for fairly and rigorously evaluating GNN based community detection, thereby potentially impeding progress in this nascent field. We observe the particular difficulties in this setting is the ambiguous hyperparameter tuning environments combined with conflicting metrics of performance and evaluation datasets. In this work, we propose and evaluate frameworks for the consistent comparisons of community detection algorithms using GNNs. With this, we show the strong dependence of the performance to the experimental settings, exacerbated by factors such as the use of GNNs and the unsupervised nature of the task, providing clear motivation for the use of a framework to facilitate congruent research in the field.

翻译：图神经网络（GNN）因其能够学习图结构上的函数（一种强大且富有表现力的数据表示形式），成为任何机器学习任务中的关键工具。社区检测作为一种无监督任务，越来越多地借助GNN来执行。利用节点特征的多维性与图的连通性对图中节点进行聚类，在从社交网络到基因组学的诸多现实应用中都发挥着重要作用。然而，目前文献中存在空白，尚未建立充分且规范的基准测试环境，用以公平且严谨地评估基于GNN的社区检测方法，这可能会阻碍这一新兴领域的发展。我们观察到，此场景下的特殊难点在于模糊的超参数调优环境，以及相互矛盾的性能指标和评估数据集。在本工作中，我们提出并评估了用于一致性比较基于GNN的社区检测算法的框架。通过此框架，我们展示了性能对实验设置的强烈依赖性，这种依赖性因GNN的使用及任务的无监督性等因素而进一步加剧，从而为采用框架促进该领域协同研究提供了明确动机。