Simulated virtual environments have been widely used to learn robotic agents that perform daily household tasks. These environments encourage research progress by far, but often provide limited object interactability, visual appearance different from real-world environments, or relatively smaller environment sizes. This prevents the learned models in the virtual scenes from being readily deployable. To bridge the gap between these learning environments and deploying (i.e., real) environments, we propose the ReALFRED benchmark that employs real-world scenes, objects, and room layouts to learn agents to complete household tasks by understanding free-form language instructions and interacting with objects in large, multi-room and 3D-captured scenes. Specifically, we extend the ALFRED benchmark with updates for larger environmental spaces with smaller visual domain gaps. With ReALFRED, we analyze previously crafted methods for the ALFRED benchmark and observe that they consistently yield lower performance in all metrics, encouraging the community to develop methods in more realistic environments. Our code and data are publicly available.

翻译：模拟虚拟环境已被广泛用于学习执行日常家务任务的机器人智能体。这些环境迄今极大地推动了研究进展，但通常提供的物体交互性有限、视觉外观与现实环境存在差异，或环境规模相对较小。这阻碍了在虚拟场景中学习的模型能够直接部署应用。为了弥合这些学习环境与部署（即真实）环境之间的差距，我们提出了ReALFRED基准，它采用真实世界的场景、物体和房间布局，通过理解自由形式的语言指令并在大型、多房间、三维捕捉的场景中与物体交互，来学习智能体完成家务任务。具体而言，我们在ALFRED基准的基础上进行了扩展，更新为具有更小视觉域差距的更大环境空间。利用ReALFRED，我们分析了先前为ALFRED基准设计的方法，并观察到这些方法在所有指标上均持续表现出更低的性能，这鼓励研究社区在更真实的环境中开发新方法。我们的代码和数据已公开提供。