Large Language Model (LLM) agents significantly extend the capabilities of standalone LLMs, empowering them to interact with external tools (e.g., APIs, functions) and complete complex tasks in a self-directed fashion. The challenge of tool use demands that LLMs not only understand user queries and generate answers but also excel in task planning, memory management, tool invocation, and result summarization. While traditional approaches focus on training a single LLM with all these capabilities, performance limitations become apparent, particularly with smaller models. Moreover, the entire LLM may require retraining when tools are updated. To overcome these challenges, we propose a novel strategy that decomposes the aforementioned capabilities into a planner, caller, and summarizer. Each component is implemented by a single LLM that focuses on a specific capability and collaborates with other components to accomplish the task. This modular framework facilitates individual updates and the potential use of smaller LLMs for building each capability. To effectively train this framework, we introduce a two-stage training paradigm. First, we fine-tune a backbone LLM on the entire dataset without discriminating sub-tasks, providing the model with a comprehensive understanding of the task. Second, the fine-tuned LLM is used to instantiate the planner, caller, and summarizer respectively, which are continually fine-tuned on respective sub-tasks. Evaluation across various tool-use benchmarks illustrates that our proposed multi-LLM framework surpasses the traditional single-LLM approach, highlighting its efficacy and advantages in tool learning.

翻译：大型语言模型（LLM）智能体显著扩展了独立LLM的能力，使其能够与外部工具（如API、函数）交互，并以自主方式完成复杂任务。工具使用这一挑战不仅要求LLM理解用户查询并生成答案，还需在任务规划、记忆管理、工具调用及结果总结方面表现出色。传统方法专注于训练一个具备所有能力的单一LLM，但性能局限逐渐显现，尤其是在小型模型中。此外，当工具更新时，整个LLM可能需要重新训练。为应对这些挑战，我们提出一种新策略，将上述能力分解为规划器、调用器和总结器。每个组件由专注于特定能力的单一LLM实现，并与其他组件协作完成任务。这种模块化框架便于单独更新，并且可以用更小型的LLM构建各能力模块。为有效训练该框架，我们引入两阶段训练范式：首先，不区分子任务，对整个数据集微调一个骨干LLM，使其全面理解任务；随后，用该微调LLM分别实例化规划器、调用器和总结器，并对各子任务持续微调。在多种工具使用基准上的评估表明，我们提出的多LLM框架优于传统单LLM方法，凸显了其在工具学习中的有效性与优势。