Autonomous agents are able to draw on a wide variety of potential sources of task knowledge; however current approaches invariably focus on only one or two. Here we investigate the challenges and impact of exploiting diverse knowledge sources to learn online, in one-shot, new tasks for a simulated office mobile robot. The resulting agent, developed in the Soar cognitive architecture, uses the following sources of domain and task knowledge: interaction with the environment, task execution and search knowledge, human natural language instruction, and responses retrieved from a large language model (GPT-3). We explore the distinct contributions of these knowledge sources and evaluate the performance of different combinations in terms of learning correct task knowledge and human workload. Results show that an agent's online integration of diverse knowledge sources improves one-shot task learning overall, reducing human feedback needed for rapid and reliable task learning.

翻译：自主代理能够利用多种潜在的任务知识来源，然而当前的方法通常只关注其中的一两种。本文研究了利用多样化知识来源在线一次性学习新任务所面临的挑战及其影响，以模拟办公室移动机器人为例。基于Soar认知架构开发的代理采用了以下领域和任务知识来源：与环境交互、任务执行与搜索知识、人类自然语言指令，以及从大语言模型（GPT-3）中检索的响应。我们探讨了这些知识来源的独特贡献，并评估了不同组合在正确学习任务知识和降低人类工作量方面的性能。结果表明，代理在线整合多样化知识来源可整体提升一次性任务学习效果，并减少快速可靠任务学习所需的人类反馈。