The recent surge in research interest in applying large language models (LLMs) to decision-making tasks has flourished by leveraging the extensive world knowledge embedded in LLMs. While there is a growing demand to tailor LLMs for custom decision-making tasks, finetuning them for specific tasks is resource-intensive and may diminish the model's generalization capabilities. Moreover, state-of-the-art language models like GPT-4 and Claude are primarily accessible through API calls, with their parametric weights remaining proprietary and unavailable to the public. This scenario emphasizes the growing need for new methodologies that allow learning from agent experiences without requiring parametric updates. To address these problems, we introduce the Experiential Learning (ExpeL) agent. Our agent autonomously gathers experiences and extracts knowledge using natural language from a collection of training tasks. At inference, the agent recalls its extracted insights and past experiences to make informed decisions. Our empirical results highlight the robust learning efficacy of the ExpeL agent, indicating a consistent enhancement in its performance as it accumulates experiences. We further explore the emerging capabilities and transfer learning potential of the ExpeL agent through qualitative observations and additional experiments.

翻译：近期，将大语言模型应用于决策任务的研究兴趣激增，这主要得益于利用了大语言模型中蕴含的广泛世界知识。尽管针对定制化决策任务来调整大语言模型的需求日益增长，但针对特定任务进行微调不仅资源密集，还可能削弱模型的泛化能力。此外，像GPT-4和Claude这样的最先进语言模型主要通过API调用访问，其参数权重仍属专有且未向公众开放。这一现状凸显了对新方法论的迫切需求，即能够在无需更新模型参数的情况下，从智能体经验中学习。为解决这些问题，我们提出了经验学习智能体。我们的智能体能够自主收集经验，并利用自然语言从一系列训练任务中提取知识。在推理阶段，智能体通过回忆其提取的见解和过往经验来做出明智决策。我们的实证结果突显了ExpeL智能体强大的学习效能，表明其性能随着经验积累而持续提升。我们进一步通过定性观察和补充实验，探讨了ExpeL智能体涌现的能力及其迁移学习潜力。