Large language models (LLMs), such as GPT4 and LLaMA, are creating significant advancements in natural language processing, due to their strong text encoding/decoding ability and newly found emergent capability (e.g., reasoning). While LLMs are mainly designed to process pure texts, there are many real-world scenarios where text data is associated with rich structure information in the form of graphs (e.g., academic networks, and e-commerce networks) or scenarios where graph data is paired with rich textual information (e.g., molecules with descriptions). Besides, although LLMs have shown their pure text-based reasoning ability, it is underexplored whether such ability can be generalized to graphs (i.e., graph-based reasoning). In this paper, we provide a systematic review of scenarios and techniques related to large language models on graphs. We first summarize potential scenarios of adopting LLMs on graphs into three categories, namely pure graphs, text-attributed graphs, and text-paired graphs. We then discuss detailed techniques for utilizing LLMs on graphs, including LLM as Predictor, LLM as Encoder, and LLM as Aligner, and compare the advantages and disadvantages of different schools of models. Furthermore, we discuss the real-world applications of such methods and summarize open-source codes and benchmark datasets. Finally, we conclude with potential future research directions in this fast-growing field. The related source can be found at https://github.com/PeterGriffinJin/Awesome-Language-Model-on-Graphs.

翻译：大型语言模型（LLMs），如GPT4和LLaMA，凭借其强大的文本编码/解码能力及新发现的涌现能力（例如推理），正在自然语言处理领域取得显著进展。尽管LLMs主要设计用于处理纯文本，但在许多现实场景中，文本数据与以图形式存在的丰富结构信息相关联（如学术网络、电子商务网络），或者图数据与丰富的文本信息配对（如带有描述的分子的场景）。此外，虽然LLMs已展现出其基于纯文本的推理能力，但这种能力能否推广到图（即基于图的推理）仍待探索。本文系统梳理了与图上大型语言模型相关的场景和技术。我们首先将LLMs在图上的潜在应用场景归纳为三类：纯图、文本属性图和文本配对图。随后，我们详细讨论了在图数据上利用LLMs的技术，包括LLM作为预测器、LLM作为编码器和LLM作为对齐器，并比较了不同模型范式的优缺点。此外，我们探讨了这类方法的实际应用，并总结了开源代码和基准数据集。最后，我们对这一快速发展领域的未来研究方向进行了展望。相关资源可访问 https://github.com/PeterGriffinJin/Awesome-Language-Model-on-Graphs。