This paper studies learning on text-attributed graphs (TAGs), where each node is associated with a text description. An ideal solution for such a problem would be integrating both the text and graph structure information with large language models and graph neural networks (GNNs). However, the problem becomes very challenging when graphs are large due to the high computational complexity brought by training large language models and GNNs together. In this paper, we propose an efficient and effective solution to learning on large text-attributed graphs by fusing graph structure and language learning with a variational Expectation-Maximization (EM) framework, called GLEM. Instead of simultaneously training large language models and GNNs on big graphs, GLEM proposes to alternatively update the two modules in the E-step and M-step. Such a procedure allows training the two modules separately while simultaneously allowing the two modules to interact and mutually enhance each other. Extensive experiments on multiple data sets demonstrate the efficiency and effectiveness of the proposed approach.

翻译：本文研究文本属性图（TAGs）的学习问题，其中每个节点关联一段文本描述。解决该问题的理想方案是将文本与图结构信息结合，并利用大型语言模型和图神经网络（GNNs）进行处理。然而，当图规模较大时，由于同时训练大型语言模型和GNNs会带来极高的计算复杂度，问题变得极具挑战性。为此，本文提出一种基于变分期望最大化（EM）框架的高效且有效的解决方案——GLEM，用于融合图结构与语言学习进行大规模文本属性图学习。与在大型图上同时训练大型语言模型和GNNs不同，GLEM在E步和M步中交替更新两个模块。这种流程允许分别训练两个模块，同时使两者能够相互交互并彼此增强。在多个数据集上的大量实验证明了所提方法的效率与有效性。