The ability to adapt to physical actions and constraints in an environment is crucial for embodied agents (e.g., robots) to effectively collaborate with humans. Such physically grounded human-AI collaboration must account for the increased complexity of the continuous state-action space and constrained dynamics caused by physical constraints. However, most existing collaboration benchmarks are discrete or do not consider physical attributes and constraints. To address this, we introduce Moving Out, a human-AI collaboration benchmark that resembles a wide range of collaboration modes affected by physical attributes and constraints, such as moving heavy items together and coordinating actions to move an item around a corner. Moving Out consists of two challenges and human-human interaction data to comprehensively evaluate models' abilities to adapt to diverse human behaviors and unseen physical attributes. To give embodied agents the capability to collaborate with humans under physical attributes and constraints, we propose a novel method, BASS (Behavior Augmentation, Simulation, and Selection), to enhance the diversity of agents and their understanding of the outcome of actions. We systematically compare BASS and state-of-the-art models in AI-AI and human-AI experiments, showing that BASS can effectively collaborate with both unseen AI and humans. The project page is available at https://live-robotics-uva.github.io/movingout_ai/.

翻译：摘要：适应环境中的物理行为与约束能力，对于具身智能体（如机器人）与人类开展有效协作至关重要。这种基于物理的人机协作必须考虑连续状态-动作空间增加的复杂性，以及物理约束导致的受限动力学特性。然而，现有多数协作基准测试要么是离散的，要么未考虑物理属性与约束。为此，我们提出Moving Out——一种人机协作基准测试，它模拟了受物理属性和约束影响的多种协作模式，例如共同搬运重物、协调动作绕过拐角移动物品等。Moving Out包含两个挑战任务与人类交互数据，可全面评估模型适应不同人类行为及未见物理属性的能力。为赋予具身智能体在物理属性和约束下与人类协作的能力，我们提出新方法BASS（行为增强、模拟与选择），以提升智能体的多样性及其对行动结果的理解。我们通过AI-AI与人类-AI实验系统对比了BASS与最先进模型，证明BASS能有效与未见过的AI及人类进行协作。项目页面详见 https://live-robotics-uva.github.io/movingout_ai/。