Researchers and practitioners have recently reframed powerful Large Language Models (LLMs) as agents, enabling them to automate complex tasks largely via the use of specialized functions. To facilitate the development of LLM agents, we present a novel paradigm of training LLM agents without modifying the LLM weights, which is particularly useful when the LLMs are difficult or inaccessible for modifications. Inspired by how humans continuously forge tools to adapt to real-world tasks, rather than change our biological structure to fit a static set of tools, we propose to progressively forge agent's functions to better solve the downstream tasks instead of modifying the LLM weights. By treating the functions as learnable `agent parameters' and leveraging the fundamental idea of model training in artificial intelligence, we develop AgentOptimizer that employs the LLM to update agents' functions and devise an agent training algorithm with two strategies, roll-back, and early-stop, to streamline the training process. With extensive experiments, we showcase that the agent training paradigm could significantly improve the performance of representative LLM agents in various downstream tasks. We also study the behavior of the agent training regarding aspects like the learning curve and domain transferability.

翻译：研究人员和实践者近期将强大的大型语言模型重新定义为智能体，使其能够主要通过专用函数的调用来自动完成复杂任务。为促进大语言模型智能体的开发，我们提出了一种无需修改大语言模型权重的智能体训练新范式，该方法特别适用于难以修改或无法访问的大语言模型场景。受人类持续打造工具以适应现实任务（而非改变生物结构以适应固定工具集）这一现象的启发，我们提出通过渐进式锻造智能体的函数集来优化下游任务解决能力，而非修改大语言模型权重。通过将函数视为可学习的"智能体参数"，并借鉴人工智能中模型训练的核心思想，我们开发了利用大语言模型更新智能体函数的AgentOptimizer工具，并设计了一套包含回滚和早停两种策略的智能体训练算法以简化训练流程。大量实验表明，该智能体训练范式能显著提升代表性大语言模型智能体在各类下游任务中的表现。我们还从学习曲线和领域迁移性等角度深入研究了智能体训练的行为特性。