Task planning is emerging as an important research topic alongside the development of large language models (LLMs). It aims to break down complex user requests into solvable sub-tasks, thereby fulfilling the original requests. In this context, the sub-tasks can be naturally viewed as a graph, where the nodes represent the sub-tasks, and the edges denote the dependencies among them. Consequently, task planning is a decision-making problem that involves selecting a connected path or subgraph within the corresponding graph and invoking it. In this paper, we explore graph learning-based methods for task planning, a direction that is orthogonal to the prevalent focus on prompt design. Our interest in graph learning stems from a theoretical discovery: the biases of attention and auto-regressive loss impede LLMs' ability to effectively navigate decision-making on graphs, which is adeptly addressed by graph neural networks (GNNs). This theoretical insight led us to integrate GNNs with LLMs to enhance overall performance. Extensive experiments demonstrate that GNN-based methods surpass existing solutions even without training, and minimal training can further enhance their performance. Additionally, our approach complements prompt engineering and fine-tuning techniques, with performance further enhanced by improved prompts or a fine-tuned model.

翻译：随着大语言模型（LLMs）的发展，任务规划正成为一个重要的研究课题。其目标是将复杂的用户请求分解为可解决的子任务，从而满足原始请求。在此背景下，子任务可以自然地视为一个图，其中节点代表子任务，边表示它们之间的依赖关系。因此，任务规划是一个决策问题，涉及在对应图中选择一条连通路径或子图并调用它。在本文中，我们探索了基于图学习的任务规划方法，这一方向与当前普遍关注的提示设计正交。我们对图学习的兴趣源于一个理论发现：注意力机制和自回归损失的偏差阻碍了LLMs在图上有效进行决策的能力，而图神经网络（GNNs）能巧妙地解决这一问题。这一理论洞见促使我们将GNNs与LLMs结合以提升整体性能。大量实验表明，基于GNN的方法即使未经训练也超越了现有解决方案，而少量训练能进一步提升其性能。此外，我们的方法可与提示工程和微调技术互补，通过改进提示或微调模型能进一步增强性能。