Intelligent instruction-following robots capable of improving from autonomously collected experience have the potential to transform robot learning: instead of collecting costly teleoperated demonstration data, large-scale deployment of fleets of robots can quickly collect larger quantities of autonomous data that can collectively improve their performance. However, autonomous improvement requires solving two key problems: (i) fully automating a scalable data collection procedure that can collect diverse and semantically meaningful robot data and (ii) learning from non-optimal, autonomous data with no human annotations. To this end, we propose a novel approach that addresses these challenges, allowing instruction-following policies to improve from autonomously collected data without human supervision. Our framework leverages vision-language models to collect and evaluate semantically meaningful experiences in new environments, and then utilizes a decomposition of instruction following tasks into (semantic) language-conditioned image generation and (non-semantic) goal reaching, which makes it significantly more practical to improve from this autonomously collected data without any human annotations. We carry out extensive experiments in the real world to demonstrate the effectiveness of our approach, and find that in a suite of unseen environments, the robot policy can be improved 2x with autonomously collected data. We open-source the code for our semantic autonomous improvement pipeline, as well as our autonomous dataset of 30.5K trajectories collected across five tabletop environments.

翻译：能够通过自主收集的经验进行改进的智能指令跟随机器人具有变革机器人学习的潜力：无需收集成本高昂的遥操作演示数据，大规模部署的机器人集群可以快速收集更大量的自主数据，从而共同提升其性能。然而，自主改进需要解决两个关键问题：(i) 实现可扩展数据收集流程的完全自动化，以获取多样化且具有语义意义的机器人数据；(ii) 从无人为标注的非最优自主数据中进行学习。为此，我们提出了一种新颖方法应对这些挑战，使指令跟随策略能够在无人监督的情况下通过自主收集的数据实现改进。我们的框架利用视觉-语言模型在新环境中收集和评估具有语义意义的经验，随后将指令跟随任务分解为（语义层面的）语言条件图像生成和（非语义层面的）目标抵达两个子任务。这种分解方式使得从无人工标注的自主收集数据中实现性能提升变得显著更可行。我们在现实世界开展了大量实验以验证方法的有效性，发现在一系列未见环境中，机器人策略通过自主收集的数据可实现2倍的性能提升。我们开源了语义自主改进流程的代码，以及包含五个桌面环境中收集的30.5K条轨迹的自主数据集。