The recent surge in research interest in applying large language models (LLMs) to decision-making tasks has flourished by leveraging the extensive world knowledge embedded in LLMs. While there is a growing demand to tailor LLMs for custom decision-making tasks, finetuning them for specific tasks is resource-intensive and may diminish the model's generalization capabilities. Moreover, state-of-the-art language models like GPT-4 and Claude are primarily accessible through API calls, with their parametric weights remaining proprietary and unavailable to the public. This scenario emphasizes the growing need for new methodologies that allow learning from agent experiences without requiring parametric updates. To address these problems, we introduce the Experiential Learning (ExpeL) agent. Our agent autonomously gathers experiences and extracts knowledge using natural language from a collection of training tasks. At inference, the agent recalls its extracted insights and past experiences to make informed decisions. Our empirical results highlight the robust learning efficacy of the ExpeL agent, indicating a consistent enhancement in its performance as it accumulates experiences. We further explore the emerging capabilities and transfer learning potential of the ExpeL agent through qualitative observations and additional experiments.

翻译：近年来，将大型语言模型（LLMs）应用于决策任务的研究兴趣激增，这得益于LLMs中蕴含的广泛世界知识。尽管针对定制决策任务调整LLMs的需求日益增长，但为特定任务微调模型不仅消耗大量资源，还可能削弱模型的泛化能力。此外，GPT-4和Claude等最先进的语言模型主要通过API调用访问，其参数权重仍是专有且不对公众开放。这一现状凸显了对新方法论的需求——允许从代理经验中学习而无需参数更新。为解决这些问题，我们引入了经验学习（ExpeL）代理。该代理自主收集经验，并从一组训练任务中通过自然语言提取知识。在推理阶段，代理会调用其提取的见解与过往经验以做出明智决策。实证结果表明，ExpeL代理具有强大的学习效能，其性能随经验积累持续提升。我们进一步通过定性观察与附加实验，探讨了ExpeL代理的新兴能力与迁移学习潜力。