Manipulating objects without grasping them enables more complex tasks, known as non-prehensile manipulation. Most previous methods only learn one manipulation skill, such as reach or push, and cannot achieve flexible object manipulation.In this work, we introduce MRLM, a Multi-stage Reinforcement Learning approach for non-prehensile Manipulation of objects.MRLM divides the task into multiple stages according to the switching of object poses and contact points.At each stage, the policy takes the point cloud-based state-goal fusion representation as input, and proposes a spatially-continuous action that including the motion of the parallel gripper pose and opening width.To fully unlock the potential of MRLM, we propose a set of technical contributions including the state-goal fusion representation, spatially-reachable distance metric, and automatic buffer compaction.We evaluate MRLM on an Occluded Grasping task which aims to grasp the object in configurations that are initially occluded.Compared with the baselines, the proposed technical contributions improve the success rate by at least 40\% and maximum 100\%, and avoids falling into local optimum.Our method demonstrates strong generalization to unseen object with shapes outside the training distribution.Moreover, MRLM can be transferred to real world with zero-shot transfer, achieving a 95\% success rate.Code and videos can be found at https://sites.google.com/view/mrlm.

翻译：不依赖抓取操控物体能够实现更复杂的任务，即非抓取操控。以往方法大多仅学习单一操控技能（如触及或推），无法实现灵活的物体操控。本文提出MRLM——一种面向物体非抓取操控的多阶段强化学习方法。MRLM根据物体姿态和接触点的切换将任务划分为多个阶段。在每个阶段，策略以基于点云的状态-目标融合表示作为输入，输出包含平行夹爪位姿和开合宽度的空间连续动作。为充分释放MRLM潜力，我们提出一套技术贡献，包括状态-目标融合表示、空间可达距离度量及自动缓存压缩。我们在遮挡抓取任务（旨在抓取初始状态下被遮挡的物体）上评估MRLM。与基线相比，所提技术贡献将成功率提升至少40%至最高100%，并避免陷入局部最优。方法展现出对训练分布外形状未知物体的强泛化能力。此外，MRLM可通过零样本迁移部署至真实世界，实现95%的成功率。代码与视频见https://sites.google.com/view/mrlm。