Self-supervised learning with masked autoencoders has recently gained popularity for its ability to produce effective image or textual representations, which can be applied to various downstream tasks without retraining. However, we observe that the current masked autoencoder models lack good generalization ability on graph data. To tackle this issue, we propose a novel graph masked autoencoder framework called GiGaMAE. Different from existing masked autoencoders that learn node presentations by explicitly reconstructing the original graph components (e.g., features or edges), in this paper, we propose to collaboratively reconstruct informative and integrated latent embeddings. By considering embeddings encompassing graph topology and attribute information as reconstruction targets, our model could capture more generalized and comprehensive knowledge. Furthermore, we introduce a mutual information based reconstruction loss that enables the effective reconstruction of multiple targets. This learning objective allows us to differentiate between the exclusive knowledge learned from a single target and common knowledge shared by multiple targets. We evaluate our method on three downstream tasks with seven datasets as benchmarks. Extensive experiments demonstrate the superiority of GiGaMAE against state-of-the-art baselines. We hope our results will shed light on the design of foundation models on graph-structured data. Our code is available at: https://github.com/sycny/GiGaMAE.

翻译：基于掩码自编码器的自监督学习因其无需重新训练即可生成有效图像或文本表示、并应用于多种下游任务的能力而近年备受关注。然而，我们观察到当前掩码自编码器模型在图数据上缺乏良好的泛化能力。为解决这一问题，我们提出了一种新颖的图掩码自编码器框架GiGaMAE。与现有通过显式重建原始图组件（如特征或边）来学习节点表示的掩码自编码器不同，本文提出协作重建信息丰富且完整的潜嵌入。通过将包含图拓扑与属性信息的嵌入作为重建目标，我们的模型能够捕获更通用、更全面的知识。此外，我们引入一种基于互信息的重建损失函数，使模型能够有效重建多个目标。该学习目标使我们能够区分从单一目标学习到的独有知识与多目标共享的通用知识。我们在七个数据集上对三个下游任务进行了评估。大量实验表明，GiGaMAE相较于最先进的基线方法具有优越性。我们期望本研究结果能为图结构数据基础模型的设计提供启示。我们的代码已公开于：https://github.com/sycny/GiGaMAE。