In order to develop robots that can effectively serve as versatile and capable home assistants, it is crucial for them to reliably perceive and interact with a wide variety of objects across diverse environments. To this end, we proposed Open Vocabulary Mobile Manipulation as a key benchmark task for robotics: finding any object in a novel environment and placing it on any receptacle surface within that environment. We organized a NeurIPS 2023 competition featuring both simulation and real-world components to evaluate solutions to this task. Our baselines on the most challenging version of this task, using real perception in simulation, achieved only an 0.8% success rate; by the end of the competition, the best participants achieved an 10.8\% success rate, a 13x improvement. We observed that the most successful teams employed a variety of methods, yet two common threads emerged among the best solutions: enhancing error detection and recovery, and improving the integration of perception with decision-making processes. In this paper, we detail the results and methodologies used, both in simulation and real-world settings. We discuss the lessons learned and their implications for future research. Additionally, we compare performance in real and simulated environments, emphasizing the necessity for robust generalization to novel settings.

翻译：为开发能够有效充当多功能家庭助手的机器人，使其能够在多样化环境中可靠感知并与广泛物体进行交互至关重要。为此，我们提出开放词汇移动操作作为机器人学的关键基准任务：在陌生环境中定位任意物体，并将其放置于该环境中的任意承载表面。我们组织了包含仿真与实体机器人组件的NeurIPS 2023竞赛以评估该任务的解决方案。在最具挑战性的真实感知仿真任务版本中，我们的基线模型仅获得0.8%的成功率；竞赛结束时，最佳参赛团队实现了10.8%的成功率，提升幅度达13倍。我们观察到最成功的团队采用了多样化方法，但最佳解决方案中呈现出两个共同特征：强化错误检测与恢复机制，以及改进感知与决策过程的融合。本文详细阐述了仿真与实体环境中的实验结果与方法体系，探讨了所获经验及其对未来研究的启示。此外，我们对比了实体与仿真环境中的性能表现，强调了对新场景实现鲁棒泛化的必要性。