Reinforcement learning often requires millions of examples to produce static, black-box models. In contrast, interactive task learning (ITL) emphasizes incremental knowledge acquisition from limited instruction provided by humans in modalities such as natural language. However, in practice, ITL systems often suffers from brittle, error-prone language parsing. Large language models (LLMs) are resistant to brittleness but are not interpretable and cannot learn incrementally. We present VAL, an ITL system with a new philosophy for LLM/symbolic integration. By using LLMs only for specific tasks -- such as predicate and argument selection -- within an algorithmic framework, VAL reaps the benefits of LLMs to support interactive learning of hierarchical task knowledge from natural language. Acquired knowledge is human interpretable and generalizes to support execution of novel tasks without additional training. We studied users' interactions with VAL in a video game setting, finding that most users could successfully teach VAL using language they felt was natural.

翻译：强化学习通常需要数百万个示例才能生成静态的黑盒模型。相比之下，交互式任务学习（ITL）强调通过自然语言等人类交互方式，从有限指令中逐步获取知识。然而在实践中，ITL系统常面临语言解析脆弱、易出错的问题。大型语言模型（LLM）虽然抗脆弱性强，但缺乏可解释性且无法增量学习。我们提出VAL——一种采用全新LLM/符号集成理念的ITL系统。通过将LLM仅用于算法框架内的特定任务（如谓词和参数选择），VAL充分利用LLM的优势，支持通过自然语言分层交互学习任务知识。所获取的知识不仅具备人类可解释性，还能泛化支持执行新任务而无需额外训练。我们在视频游戏场景中研究了用户与VAL的交互，发现大多数用户能够使用他们认为自然的语言成功教会VAL。