Recently, pretraining methods for the Graph Neural Networks (GNNs) have been successful at learning effective representations from unlabeled graph data. However, most of these methods rely on pairwise relations in the graph and do not capture the underling higher-order relations between entities. Hypergraphs are versatile and expressive structures that can effectively model higher-order relationships among entities in the data. Despite the efforts to adapt GNNs to hypergraphs (HyperGNN), there are currently no fully self-supervised pretraining methods for HyperGNN on heterogeneous hypergraphs. In this paper, we present SPHH, a novel self-supervised pretraining framework for heterogeneous HyperGNNs. Our method is able to effectively capture higher-order relations among entities in the data in a self-supervised manner. SPHH is consist of two self-supervised pretraining tasks that aim to simultaneously learn both local and global representations of the entities in the hypergraph by using informative representations derived from the hypergraph structure. Overall, our work presents a significant advancement in the field of self-supervised pretraining of HyperGNNs, and has the potential to improve the performance of various graph-based downstream tasks such as node classification and link prediction tasks which are mapped to hypergraph configuration. Our experiments on two real-world benchmarks using four different HyperGNN models show that our proposed SPHH framework consistently outperforms state-of-the-art baselines in various downstream tasks. The results demonstrate that SPHH is able to improve the performance of various HyperGNN models in various downstream tasks, regardless of their architecture or complexity, which highlights the robustness of our framework.

翻译：近来，图神经网络（GNNs）的预训练方法在从无标签图数据中学习有效表征方面取得了成功。然而，这些方法大多依赖于图中的成对关系，未能捕捉实体间潜在的更高阶关系。超图是一种通用且富有表达力的结构，能够有效建模数据中实体间的高阶关系。尽管已有将GNN适配至超图（HyperGNN）的努力，但目前尚无完全自监督的预训练方法适用于异构超图上的HyperGNN。本文提出SPHH——一种针对异构HyperGNN的新型自监督预训练框架。该方法能够以自监督方式有效捕捉数据中实体间的高阶关系。SPHH包含两个自监督预训练任务，旨在通过利用超图结构衍生的信息表征，同时学习超图中实体的局部与全局表征。总体而言，我们的工作推动了HyperGNN自监督预训练领域的重大进展，并有望提升多种基于图的下游任务（如映射至超图配置的节点分类与链接预测任务）的性能。我们在两个真实世界基准上使用四种不同HyperGNN模型进行的实验表明，所提出的SPHH框架在多种下游任务中持续优于最先进的基线方法。结果证明，无论其架构或复杂度如何，SPHH都能提升各类HyperGNN模型在多种下游任务中的性能，凸显了本框架的鲁棒性。