Intelligent instruction-following robots capable of improving from autonomously collected experience have the potential to transform robot learning: instead of collecting costly teleoperated demonstration data, large-scale deployment of fleets of robots can quickly collect larger quantities of autonomous data that can collectively improve their performance. However, autonomous improvement requires solving two key problems: (i) fully automating a scalable data collection procedure that can collect diverse and semantically meaningful robot data and (ii) learning from non-optimal, autonomous data with no human annotations. To this end, we propose a novel approach that addresses these challenges, allowing instruction-following policies to improve from autonomously collected data without human supervision. Our framework leverages vision-language models to collect and evaluate semantically meaningful experiences in new environments, and then utilizes a decomposition of instruction following tasks into (semantic) language-conditioned image generation and (non-semantic) goal reaching, which makes it significantly more practical to improve from this autonomously collected data without any human annotations. We carry out extensive experiments in the real world to demonstrate the effectiveness of our approach, and find that in a suite of unseen environments, the robot policy can be improved significantly with autonomously collected data. We open-source the code for our semantic autonomous improvement pipeline, as well as our autonomous dataset of 30.5K trajectories collected across five tabletop environments.

翻译：能够从自主收集的经验中改进的智能指令跟随机器人具有变革机器人学习的潜力：无需收集成本高昂的遥操作演示数据，大规模部署机器人集群可快速采集更大量的自主数据，从而整体提升其性能。然而，自主改进需要解决两个关键问题：(i) 实现可扩展数据收集流程的完全自动化，以获取多样化且具有语义意义的机器人数据；(ii) 从无人为标注的非最优自主数据中进行学习。为此，我们提出一种新颖方法应对这些挑战，使指令跟随策略能够在无人监督的情况下从自主收集的数据中改进。我们的框架利用视觉语言模型在新环境中收集和评估具有语义意义的经验，随后将指令跟随任务分解为（语义）语言条件图像生成与（非语义）目标抵达两个部分，这使得在无需任何人工标注的情况下，从自主收集的数据中进行改进变得显著更易实现。我们在现实世界开展了大量实验以验证方法的有效性，发现在一系列未见环境中，机器人策略可通过自主收集的数据获得显著提升。我们开源了语义自主改进流程的代码，以及在五个桌面环境中收集的包含30.5K条轨迹的自主数据集。