How do we imbue robots with the ability to efficiently manipulate unseen objects and transfer relevant skills based on demonstrations? End-to-end learning methods often fail to generalize to novel objects or unseen configurations. Instead, we focus on the task-specific pose relationship between relevant parts of interacting objects. We conjecture that this relationship is a generalizable notion of a manipulation task that can transfer to new objects in the same category; examples include the relationship between the pose of a pan relative to an oven or the pose of a mug relative to a mug rack. We call this task-specific pose relationship "cross-pose" and provide a mathematical definition of this concept. We propose a vision-based system that learns to estimate the cross-pose between two objects for a given manipulation task using learned cross-object correspondences. The estimated cross-pose is then used to guide a downstream motion planner to manipulate the objects into the desired pose relationship (placing a pan into the oven or the mug onto the mug rack). We demonstrate our method's capability to generalize to unseen objects, in some cases after training on only 10 demonstrations in the real world. Results show that our system achieves state-of-the-art performance in both simulated and real-world experiments across a number of tasks. Supplementary information and videos can be found at https://sites.google.com/view/tax-pose/home.

翻译：如何赋予机器人基于示范高效操作未见物体并迁移相关技能的能力？端到端学习方法通常难以泛化至新物体或未见配置。为此，我们聚焦于交互物体相关部件之间的任务特定姿态关系。我们推测这种关系是操作任务的一种可泛化概念，能够迁移至同一类别的新物体；例如，锅相对于烤箱的位姿，或杯子相对于杯架的位姿。我们将这种任务特定姿态关系称为“交叉姿态”，并给出该概念的数学定义。我们提出一种基于视觉的系统，通过学习物体间跨对象对应关系，为给定操作任务估计两个物体之间的交叉姿态。随后，利用估计的交叉姿态引导下游运动规划器，将物体操纵至目标姿态关系（如将锅放入烤箱或将杯子挂至杯架）。我们证明了该方法在真实世界中仅训练10次示范后，即可泛化至未见物体。实验结果表明，该系统在多项任务的仿真与现实实验中均达到了最先进水平。补充信息与视频请见https://sites.google.com/view/tax-pose/home。