Manipulating objects without grasping them is an essential component of human dexterity, referred to as non-prehensile manipulation. Non-prehensile manipulation may enable more complex interactions with the objects, but also presents challenges in reasoning about gripper-object interactions. In this work, we introduce Hybrid Actor-Critic Maps for Manipulation (HACMan), a reinforcement learning approach for 6D non-prehensile manipulation of objects using point cloud observations. HACMan proposes a temporally-abstracted and spatially-grounded object-centric action representation that consists of selecting a contact location from the object point cloud and a set of motion parameters describing how the robot will move after making contact. We modify an existing off-policy RL algorithm to learn in this hybrid discrete-continuous action representation. We evaluate HACMan on a 6D object pose alignment task in both simulation and in the real world. On the hardest version of our task, with randomized initial poses, randomized 6D goals, and diverse object categories, our policy demonstrates strong generalization to unseen object categories without a performance drop, achieving an 89% success rate on unseen objects in simulation and 50% success rate with zero-shot transfer in the real world. Compared to alternative action representations, HACMan achieves a success rate more than three times higher than the best baseline. With zero-shot sim2real transfer, our policy can successfully manipulate unseen objects in the real world for challenging non-planar goals, using dynamic and contact-rich non-prehensile skills. Videos can be found on the project website: https://hacman-2023.github.io.

翻译：在不抓握的情况下操作物体是人类灵巧性的重要组成部分，称为非抓取操作。非抓取操作可能实现与物体更复杂的交互，但也对夹爪-物体相互作用的推理提出了挑战。本研究提出用于操作的混合行动者-评论家映射（HACMan），一种利用点云观测进行物体六自由度非抓取操作的强化学习方法。HACMan提出了一种时间抽象且空间接地的以物体为中心的动作表示，包括从物体点云中选择接触位置，以及描述机器人在接触后如何运动的一组运动参数。我们改进现有的离策略强化学习算法，以学习这种混合离散-连续动作表示。我们在仿真和现实世界中评估HACMan在六自由度物体姿态对齐任务上的表现。在我们任务的最难版本中（包含随机初始姿态、随机六自由度目标及多样化物体类别），我们的策略在未见物体类别上表现出强大的泛化能力且性能无下降，在仿真中对未见物体达到89%的成功率，在现实世界中通过零样本迁移达到50%的成功率。与替代动作表示相比，HACMan的成功率比最佳基线高出三倍以上。通过零样本仿真到现实迁移，我们的策略能够在现实世界中成功操作未见物体以实现具有挑战性的非平面目标，并运用动态且接触丰富的非抓取技能。视频可在项目网站查看：https://hacman-2023.github.io。