Behavior cloning has shown promise for robot manipulation, but real-world demonstrations are costly to acquire at scale. While simulated data offers a scalable alternative, particularly with advances in automated demonstration generation, transferring policies to the real world is hampered by various simulation and real domain gaps. In this work, we propose a unified sim-and-real co-training framework for learning generalizable manipulation policies that primarily leverages simulation and only requires a few real-world demonstrations. Central to our approach is learning a domain-invariant, task-relevant feature space. Our key insight is that aligning the joint distributions of observations and their corresponding actions across domains provides a richer signal than aligning observations (marginals) alone. We achieve this by embedding an Optimal Transport (OT)-inspired loss within the co-training framework, and extend this to an Unbalanced OT framework to handle the imbalance between abundant simulation data and limited real-world examples. We validate our method on challenging manipulation tasks, showing it can leverage abundant simulation data to achieve up to a 30% improvement in the real-world success rate and even generalize to scenarios seen only in simulation. Project webpage: https://ot-sim2real.github.io/.

翻译：行为克隆在机器人操作任务中展现出潜力，但大规模获取真实世界演示数据成本高昂。尽管仿真数据提供了可扩展的替代方案（特别是在自动演示生成技术进步的背景下），但策略向真实世界的迁移仍受到多种仿真与真实领域差异的阻碍。本研究提出一种统一的仿真与真实协同训练框架，用于学习可泛化的操作策略，该框架主要利用仿真数据，仅需少量真实世界演示。我们方法的核心在于学习一个领域不变且任务相关的特征空间。关键洞见是：跨领域对齐观测数据及其对应动作的联合分布，比仅对齐观测数据（边缘分布）能提供更丰富的信号。我们通过在协同训练框架中嵌入最优传输（OT）启发的损失函数来实现这一目标，并将其扩展至非平衡OT框架以处理海量仿真数据与有限真实样本之间的不平衡问题。我们在具有挑战性的操作任务上验证了所提方法，结果表明其能够利用丰富的仿真数据将真实世界成功率提升高达30%，甚至能泛化至仅在仿真中出现过的场景。项目网页：https://ot-sim2real.github.io/。