In recent years, diffusion models have achieved remarkable success in various domains of artificial intelligence, such as image synthesis, super-resolution, and 3D molecule generation. However, the application of diffusion models in graph learning has received relatively little attention. In this paper, we address this gap by investigating the use of diffusion models for unsupervised graph representation learning. We begin by identifying the anisotropic structures of graphs and a crucial limitation of the vanilla forward diffusion process in learning anisotropic structures. This process relies on continuously adding an isotropic Gaussian noise to the data, which may convert the anisotropic signals to noise too quickly. This rapid conversion hampers the training of denoising neural networks and impedes the acquisition of semantically meaningful representations in the reverse process. To address this challenge, we propose a new class of models called {\it directional diffusion models}. These models incorporate data-dependent, anisotropic, and directional noises in the forward diffusion process. To assess the efficacy of our proposed models, we conduct extensive experiments on 12 publicly available datasets, focusing on two distinct graph representation learning tasks. The experimental results demonstrate the superiority of our models over state-of-the-art baselines, indicating their effectiveness in capturing meaningful graph representations. Our studies not only provide valuable insights into the forward process of diffusion models but also highlight the wide-ranging potential of these models for various graph-related tasks.

翻译：近年来，扩散模型在人工智能的多个领域取得了显著成功，例如图像合成、超分辨率以及三维分子生成。然而，扩散模型在图学习中的应用却相对较少受到关注。在本文中，我们通过研究扩散模型在无监督图表示学习中的应用来弥补这一空白。首先，我们识别出图的各向异性结构，并指出现有前向扩散过程在学习各向异性结构时的一个关键限制。该过程依赖于持续向数据添加各向同性高斯噪声，这可能导致各向异性信号过快转化为噪声。这种快速转换阻碍了去噪神经网络的训练，并妨碍了逆向过程中获得具有语义意义的表示。为应对这一挑战，我们提出了一类新模型——方向扩散模型。这些模型在前向扩散过程中引入了数据依赖的、各向异性的方向性噪声。为了评估所提模型的有效性，我们在12个公开数据集上开展了大量实验，聚焦于两种不同的图表示学习任务。实验结果表明，我们的模型优于当前最先进的基线方法，展示了其在捕获有意义图表示方面的有效性。我们的研究不仅为扩散模型的前向过程提供了有价值的见解，也凸显了这些模型在各类图相关任务中的广泛应用潜力。