Generative self-supervised learning (SSL) has exhibited significant potential and garnered increasing interest in graph learning. In this study, we aim to explore the problem of generative SSL in the context of heterogeneous graph learning (HGL). The previous SSL approaches for heterogeneous graphs have primarily relied on contrastive learning, necessitating the design of complex views to capture heterogeneity. However, existing generative SSL methods have not fully leveraged the capabilities of generative models to address the challenges of HGL. In this paper, we present HGCVAE, a novel contrastive variational graph auto-encoder that liberates HGL from the burden of intricate heterogeneity capturing. Instead of focusing on complicated heterogeneity, HGCVAE harnesses the full potential of generative SSL. HGCVAE innovatively consolidates contrastive learning with generative SSL, introducing several key innovations. Firstly, we employ a progressive mechanism to generate high-quality hard negative samples for contrastive learning, utilizing the power of variational inference. Additionally, we present a dynamic mask strategy to ensure effective and stable learning. Moreover, we propose an enhanced scaled cosine error as the criterion for better attribute reconstruction. As an initial step in combining generative and contrastive SSL, HGCVAE achieves remarkable results compared to various state-of-the-art baselines, confirming its superiority.

翻译：生成式自监督学习在图学习领域展现出显著潜力并引发日益增长的兴趣。本研究旨在探索异构图学习背景下生成式自监督学习的问题。以往异构图的自监督学习方法主要依赖对比学习，需设计复杂视图以捕捉异质性，但现有生成式自监督方法尚未充分利用生成式模型应对异构图学习挑战的能力。本文提出HGCVAE——一种新颖的对比变分图自编码器，通过释放异构图学习的复杂性捕获负担，转而聚焦生成式自监督学习的全潜能开发。HGCVAE创新性地将对比学习与生成式自监督学习相结合，引入多项关键突破：首先，利用变分推断的渐进式机制生成高质量困难负样本以增强对比学习；其次，提出动态掩码策略确保有效且稳定的学习过程；此外，采用增强型缩放余弦误差作为属性重构标准。作为融合生成式与对比自监督学习的先驱性尝试，HGCVAE在多项最新基准测试中取得显著成效，充分验证了其优越性。