Large language models (LLMs), such as ChatGPT and LLaMA, are creating significant advancements in natural language processing, due to their strong text encoding/decoding ability and newly found emergent capability (e.g., reasoning). While LLMs are mainly designed to process pure texts, there are many real-world scenarios where text data are associated with rich structure information in the form of graphs (e.g., academic networks, and e-commerce networks) or scenarios where graph data are paired with rich textual information (e.g., molecules with descriptions). Besides, although LLMs have shown their pure text-based reasoning ability, it is underexplored whether such ability can be generalized to graph scenarios (i.e., graph-based reasoning). In this paper, we provide a systematic review of scenarios and techniques related to large language models on graphs. We first summarize potential scenarios of adopting LLMs on graphs into three categories, namely pure graphs, text-rich graphs, and text-paired graphs. We then discuss detailed techniques for utilizing LLMs on graphs, including LLM as Predictor, LLM as Encoder, and LLM as Aligner, and compare the advantages and disadvantages of different schools of models. Furthermore, we mention the real-world applications of such methods and summarize open-source codes and benchmark datasets. Finally, we conclude with potential future research directions in this fast-growing field. The related source can be found at https://github.com/PeterGriffinJin/Awesome-Language-Model-on-Graphs.

翻译：大语言模型（LLMs），如ChatGPT和LLaMA，凭借其强大的文本编码/解码能力以及新涌现的推理等能力，正推动自然语言处理领域取得重大进展。尽管LLMs主要设计用于处理纯文本，但在许多实际场景中，文本数据常与图结构中的丰富结构信息相关联（例如学术网络、电子商务网络），或者图数据与丰富的文本信息配对出现（例如带描述的分子）。此外，虽然LLMs已展现出基于纯文本的推理能力，但这种能力能否泛化到图场景（即基于图的推理）仍有待探索。本文系统梳理了大语言模型在图上的相关场景与技术。我们首先将LLMs在图上的潜在应用场景归纳为三类：纯图、文本丰富图和文本配对图。随后深入探讨了利用LLMs处理图的具体技术，包括LLM作为预测器、LLM作为编码器和LLM作为对齐器，并比较了不同模型学派各自的优缺点。此外，我们介绍了这些方法的实际应用，总结了开源代码与基准数据集。最后，我们对这一快速发展领域的未来潜在研究方向进行了展望。相关资源可参见 https://github.com/PeterGriffinJin/Awesome-Language-Model-on-Graphs。