Humans can imagine goal states during planning and perform actions to match those goals. In this work, we propose Imagination Policy, a novel multi-task key-frame policy network for solving high-precision pick and place tasks. Instead of learning actions directly, Imagination Policy generates point clouds to imagine desired states which are then translated to actions using rigid action estimation. This transforms action inference into a local generative task. We leverage pick and place symmetries underlying the tasks in the generation process and achieve extremely high sample efficiency and generalizability to unseen configurations. Finally, we demonstrate state-of-the-art performance across various tasks on the RLbench benchmark compared with several strong baselines and validate our approach on a real robot.

翻译：人类能够在规划过程中想象目标状态，并执行动作以实现这些目标。本文提出想象策略——一种用于解决高精度抓取放置任务的新型多任务关键帧策略网络。该策略不直接学习动作，而是通过生成点云来想象期望状态，随后通过刚性动作估计将状态转换为动作。这种方法将动作推断转化为局部生成任务。我们在生成过程中利用任务内在的抓取放置对称性，实现了极高的样本效率以及对未见配置的泛化能力。最终，我们在RLbench基准测试中展示了相较于多个强基线模型的最优性能，并在真实机器人平台上验证了该方法的有效性。