Graph Neural Networks (GNNs) are a pertinent tool for any machine learning task due to their ability to learn functions over graph structures, a powerful and expressive data representation. The detection of communities, an unsupervised task has increasingly been performed with GNNs. Clustering nodes in a graph using the multi-dimensionality of node features with the connectivity of the graph has many applications to real world tasks from social networks to genomics. Unfortunately, there is currently a gap in the literature with no established sufficient benchmarking environment for fairly and rigorously evaluating GNN based community detection, thereby potentially impeding progress in this nascent field. We observe the particular difficulties in this setting is the ambiguous hyperparameter tuning environments combined with conflicting metrics of performance and evaluation datasets. In this work, we propose and evaluate frameworks for the consistent comparisons of community detection algorithms using GNNs. With this, we show the strong dependence of the performance to the experimental settings, exacerbated by factors such as the use of GNNs and the unsupervised nature of the task, providing clear motivation for the use of a framework to facilitate congruent research in the field.

翻译：图神经网络（GNN）因其在图形结构（一种强大且富有表现力的数据表示）上学习函数的能力，成为任何机器学习任务中相关工具。社区检测这一无监督任务越来越多地使用GNN进行。利用节点特征的多维性以及图的连通性对图中的节点进行聚类，在从社交网络到基因组学的许多现实世界任务中都有应用。不幸的是，目前文献中尚存在空白，没有建立完善的基准测试环境来公平且严谨地评估基于GNN的社区检测，这可能阻碍了这一新兴领域的进展。我们观察到，这一场景下的特殊困难在于模糊的超参数调优环境，结合了相互矛盾的性能指标和评估数据集。在这项工作中，我们提出并评估了使用GNN进行社区检测算法一致性比较的框架。通过此框架，我们展示了性能对实验设置的强烈依赖性，而GNN的使用以及任务的无监督性质等因素加剧了这种依赖性，这为使用框架以促进该领域协同研究提供了明确动机。