Manipulating objects without grasping them is an essential component of human dexterity, referred to as non-prehensile manipulation. Non-prehensile manipulation may enable more complex interactions with the objects, but also presents challenges in reasoning about gripper-object interactions. In this work, we introduce Hybrid Actor-Critic Maps for Manipulation (HACMan), a reinforcement learning approach for 6D non-prehensile manipulation of objects using point cloud observations. HACMan proposes a temporally-abstracted and spatially-grounded object-centric action representation that consists of selecting a contact location from the object point cloud and a set of motion parameters describing how the robot will move after making contact. We modify an existing off-policy RL algorithm to learn in this hybrid discrete-continuous action representation. We evaluate HACMan on a 6D object pose alignment task in both simulation and in the real world. On the hardest version of our task, with randomized initial poses, randomized 6D goals, and diverse object categories, our policy demonstrates strong generalization to unseen object categories without a performance drop, achieving an 89% success rate on unseen objects in simulation and 50% success rate with zero-shot transfer in the real world. Compared to alternative action representations, HACMan achieves a success rate more than three times higher than the best baseline. With zero-shot sim2real transfer, our policy can successfully manipulate unseen objects in the real world for challenging non-planar goals, using dynamic and contact-rich non-prehensile skills. Videos can be found on the project website: https://hacman-2023.github.io.

翻译：无需抓取即可操作物体是人类灵巧性的关键组成部分，称为非抓取操作。非抓取操作能实现与物体更复杂的交互，但也在推理夹爪-物体相互作用方面带来挑战。本文提出用于操作任务的混合演员-评论家映射（HACMan），这是一种基于点云观测的六自由度非抓取物体操作强化学习方法。HACMan提出一种时间抽象且空间锚定的物体中心动作表示，其核心是从物体点云中选择接触位置，并定义描述接触后机器人运动轨迹的运动参数集合。我们改进现有离策略强化学习算法，使其适应这种混合离散-连续动作表示的学习。在仿真和真实环境中，我们基于六自由度物体姿态对齐任务评估HACMan。在任务最困难版本中（包含随机初始姿态、随机六维目标及多样化物体类别），我们的策略展现对未见物体类别的强泛化能力，性能未出现下降，仿真中对未知物体的成功率达89%，零样本迁移至真实环境的成功率达50%。相较于其他动作表示方法，HACMan的成功率比最优基线高出三倍以上。通过零样本仿真到现实迁移，我们的策略能成功操纵真实环境中未见物体完成挑战性非平面目标，运用动态且富含接触的非抓取技能。视频见项目网站：https://hacman-2023.github.io。