Masked graph autoencoder (MGAE) has emerged as a promising self-supervised graph pre-training (SGP) paradigm due to its simplicity and effectiveness. However, existing efforts perform the mask-then-reconstruct operation in the raw data space as is done in computer vision (CV) and natural language processing (NLP) areas, while neglecting the important non-Euclidean property of graph data. As a result, the highly unstable local connection structures largely increase the uncertainty in inferring masked data and decrease the reliability of the exploited self-supervision signals, leading to inferior representations for downstream evaluations. To address this issue, we propose a novel SGP method termed Robust mAsked gRaph autoEncoder (RARE) to improve the certainty in inferring masked data and the reliability of the self-supervision mechanism by further masking and reconstructing node samples in the high-order latent feature space. Through both theoretical and empirical analyses, we have discovered that performing a joint mask-then-reconstruct strategy in both latent feature and raw data spaces could yield improved stability and performance. To this end, we elaborately design a masked latent feature completion scheme, which predicts latent features of masked nodes under the guidance of high-order sample correlations that are hard to be observed from the raw data perspective. Specifically, we first adopt a latent feature predictor to predict the masked latent features from the visible ones. Next, we encode the raw data of masked samples with a momentum graph encoder and subsequently employ the resulting representations to improve predicted results through latent feature matching. Extensive experiments on seventeen datasets have demonstrated the effectiveness and robustness of RARE against state-of-the-art (SOTA) competitors across three downstream tasks.

翻译：掩码图自编码器（MGAE）因其简洁性和有效性，已成为一种极具前景的自监督图预训练（SGP）范式。然而，现有方法像计算机视觉（CV）和自然语言处理（NLP）领域一样，在原始数据空间中执行“掩码-重建”操作，却忽略了图数据重要的非欧几里得性质。这导致高度不稳定的局部连接结构大幅增加了推断掩码数据的不确定性，并降低了所利用的自监督信号的可靠性，从而使下游评估的表征质量较差。为解决此问题，我们提出一种新型SGP方法——鲁棒掩码图自编码器（RARE），通过在高阶潜在特征空间中对节点样本进行额外掩码与重建，以提高推断掩码数据的确定性和自监督机制的可靠性。通过理论与实证分析，我们发现，在潜在特征空间与原始数据空间中联合执行“掩码-重建”策略，可带来更优的稳定性与性能。为此，我们精心设计了一种掩码潜在特征补全方案，该方案在难以从原始数据视角观察到的高阶样本相关性指导下，预测被掩码节点的潜在特征。具体而言，我们首先采用潜在特征预测器，根据可见潜在特征预测被掩码的潜在特征。接着，利用动量图编码器对被掩码样本的原始数据进行编码，并通过潜在特征匹配，将所得表征用于改进预测结果。在十七个数据集上的大量实验表明，RARE在三个下游任务中相比当前最优（SOTA）方法具有更强的有效性和鲁棒性。