Objects rarely sit in isolation in human environments. As such, we'd like our robots to reason about how multiple objects relate to one another and how those relations may change as the robot interacts with the world. To this end, we propose a novel graph neural network framework for multi-object manipulation to predict how inter-object relations change given robot actions. Our model operates on partial-view point clouds and can reason about multiple objects dynamically interacting during the manipulation. By learning a dynamics model in a learned latent graph embedding space, our model enables multi-step planning to reach target goal relations. We show our model trained purely in simulation transfers well to the real world. Our planner enables the robot to rearrange a variable number of objects with a range of shapes and sizes using both push and pick and place skills.

翻译：对象在人类环境中很少被孤立。 因此, 我们想要我们的机器人来解释 多重物体彼此之间是如何联系的, 随着机器人与世界的相互作用, 这些关系会如何变化。 为此, 我们提出一个新颖的图形神经网络框架, 供多点操作使用, 以预测跨点关系如何改变机器人动作。 我们的模型在部分视图点云上运行, 并且可以在操作过程中对多个物体进行动态互动。 通过在一个学习过的隐形嵌入空间学习一个动态模型, 我们的模型使得多步规划能够达到目标关系。 我们展示了我们纯粹在模拟中训练的模型, 向真实世界转移。 我们的规划者让机器人能够利用推、 挑、 和 定位技巧来重新排列一系列形状和大小的变量对象 。