In this paper, we aim to develop a large language model (LLM) with the reasoning ability on complex graph data. Currently, LLMs have achieved very impressive performance on various natural language learning tasks, extensions of which have also been applied to study the vision tasks with multi-modal data. However, when it comes to the graph learning tasks, existing LLMs present very serious flaws due to their several inherited weaknesses in performing {multi-step logic reasoning}, {precise mathematical calculation} and {perception about the spatial and temporal factors}. To address such challenges, in this paper, we will investigate the principles, methodologies and algorithms to empower existing LLMs with graph reasoning ability, which will have tremendous impacts on the current research of both LLMs and graph learning. Inspired by the latest ChatGPT and Toolformer models, we propose the Graph-ToolFormer (Graph Reasoning oriented Toolformer) framework to teach LLMs themselves with prompts augmented by ChatGPT to use external graph reasoning API tools. Specifically, we will investigate to teach Graph-ToolFormer to handle various graph data reasoning tasks in this paper, including both (1) very basic graph data loading and graph property reasoning tasks, ranging from simple graph order and size to the graph diameter and periphery, and (2) more advanced reasoning tasks on real-world graph data, such as bibliographic networks, protein molecules, sequential recommender systems, social networks and knowledge graphs. To demonstrate the effectiveness of Graph-ToolFormer, we conduct some preliminary experimental studies on various graph reasoning datasets and tasks, and will launch a LLM demo online with various graph reasoning abilities.

翻译：本文旨在开发一种具备复杂图数据推理能力的大语言模型（LLM）。当前，LLMs已在各类自然语言学习任务中展现出卓越性能，其扩展应用也已延伸至多模态数据的视觉任务研究。然而，在面临图学习任务时，现有LLMs因在{多步逻辑推理}、{精确数学计算}和{时空因素感知}方面存在的固有问题而表现出严重缺陷。针对这些挑战，本文将研究赋予现有LLMs图推理能力的原理、方法与算法，这将深刻影响当前LLMs与图学习两个领域的研究。受最新ChatGPT与Toolformer模型启发，我们提出Graph-ToolFormer（面向图推理的Toolformer）框架，通过ChatGPT增强的提示引导LLMs自主使用外部图推理API工具。具体而言，本文将探索如何训练Graph-ToolFormer处理多种图数据推理任务，包括：（1）基础图数据加载与图属性推理任务，涵盖从简单图阶与图规模到图直径与外围等范畴；（2）基于真实世界图数据的高级推理任务，涉及文献网络、蛋白质分子、序列推荐系统、社交网络与知识图谱等场景。为验证Graph-ToolFormer的有效性，我们已在多个图推理数据集与任务上开展初步实验研究，并将上线具备多种图推理能力的LLM在线演示系统。