We introduce a new benchmark, LLF-Bench (Learning from Language Feedback Benchmark; pronounced as "elf-bench"), to evaluate the ability of AI agents to interactively learn from natural language feedback and instructions. Learning from language feedback (LLF) is essential for people, largely because the rich information this feedback provides can help a learner avoid much of trial and error and thereby speed up the learning process. Large Language Models (LLMs) have recently enabled AI agents to comprehend natural language -- and hence AI agents can potentially benefit from language feedback during learning like humans do. But existing interactive benchmarks do not assess this crucial capability: they either use numeric reward feedback or require no learning at all (only planning or information retrieval). LLF-Bench is designed to fill this omission. LLF-Bench is a diverse collection of sequential decision-making tasks that includes user recommendation, poem writing, navigation, and robot control. The objective of an agent is to interactively solve these tasks based on their natural-language instructions and the feedback received after taking actions. Crucially, to ensure that the agent actually "learns" from the feedback, LLF-Bench implements several randomization techniques (such as paraphrasing and environment randomization) to ensure that the task isn't familiar to the agent and that the agent is robust to various verbalizations. In addition, LLF-Bench provides a unified OpenAI Gym interface for all its tasks and allows the users to easily configure the information the feedback conveys (among suggestion, explanation, and instantaneous performance) to study how agents respond to different types of feedback. Together, these features make LLF-Bench a unique research platform for developing and testing LLF agents.

翻译：摘要：我们提出了一项新基准——LLF-Bench（语言反馈学习基准，读作“elf-bench”），旨在评估AI代理从自然语言反馈和指令中进行交互学习的能力。语言反馈学习（LLF）对人类至关重要，主要因为这种反馈提供的丰富信息能帮助学习者避免大量试错，从而加速学习进程。大型语言模型（LLMs）近期使AI代理能够理解自然语言，因此AI代理在学习过程中有望像人类一样从语言反馈中获益。然而，现有的交互基准并未评估这一关键能力：它们要么使用数值奖励反馈，要么完全不要求学习（仅需规划或信息检索）。LLF-Bench旨在填补这一空白。LLF-Bench是一个多样化的序列决策任务集合，包括用户推荐、诗歌写作、导航和机器人控制。代理的目标是根据自然语言指令以及执行动作后收到的反馈，以交互方式解决这些任务。关键在于，为确保代理真正从反馈中“学习”，LLF-Bench实现了多种随机化技术（如释义和环境随机化），确保任务对代理而言不熟悉，且代理对各种语言表达具有鲁棒性。此外，LLF-Bench为其所有任务提供了统一的OpenAI Gym接口，并允许用户轻松配置反馈所传达的信息（包括建议、解释和即时性能），以研究代理如何应对不同类型的反馈。这些特性共同使LLF-Bench成为开发与测试LLF代理的独特研究平台。