Manipulating objects without grasping them is an essential component of human dexterity, referred to as non-prehensile manipulation. Non-prehensile manipulation may enable more complex interactions with the objects, but also presents challenges in reasoning about the interactions. In this work, we introduce Hybrid Actor-Critic Maps for Manipulation (HACMan), a reinforcement learning approach for 6D non-prehensile manipulation of objects using point cloud observations. HACMan proposes a temporally-abstracted and spatially-grounded object-centric action representation that consists of selecting a contact location from the object point cloud and a set of motion parameters describing how the robot will move after making contact. We modify an existing off-policy RL algorithm to learn in this hybrid discrete-continuous action representation. We evaluate HACMan on a 6D object pose alignment task in both simulation and in the real world. On the hardest version of our task, with randomized initial pose, randomized 6D goals, and diverse object categories, our policy demonstrates strong generalization to unseen object categories without a performance drop, achieving a 79% success rate on non-flat objects. Compared to alternative action representations, HACMan achieves a success rate more than three times higher than the best baseline. With zero-shot sim2real transfer, our policy can successfully manipulate unseen objects in the real world for challenging non-planar goals, using dynamic and contact-rich non-prehensile skills. Videos can be found on the project website: https://hacman-2023.github.io .

翻译：非抓取操作（即不通过抓取物体进行操作）是人类灵巧操作的重要组成部分。非抓取操作虽然能与物体产生更复杂的交互，但也给交互推理带来了挑战。本文提出混合演员-评论家操作映射（HACMan），一种基于点云观测的六维非抓取操作强化学习方法。HACMan提出一种时间抽象且空间具身化的以物体为中心的动作表征，该表征包含从物体点云中选择接触位置，以及描述机器人接触后运动方式的一组运动参数。我们改进了一种现有的离策略强化学习算法，使其能够在离散-连续混合动作表征中进行学习。我们在仿真和真实环境中对HACMan进行了六维物体姿态对齐任务的评估。在任务的最难版本中（包含随机初始姿态、随机六维目标及多种物体类别），我们的策略展现出对未见物体类别的强泛化能力且性能未出现下降，对非平面物体的成功率达79%。相较于其他动作表征，HACMan的成功率比最佳基线方法高出三倍以上。通过零样本仿真到真实迁移，我们的策略能够成功地在真实环境中操作未见物体，完成具有挑战性的非平面目标，并运用动态且富含接触的非抓取技能。视频参见项目网站：https://hacman-2023.github.io。