Many practically relevant robot grasping problems feature a target object for which all grasps are occluded, e.g., by the environment. Single-shot grasp planning invariably fails in such scenarios. Instead, it is necessary to first manipulate the object into a configuration that affords a grasp. We solve this problem by learning a sequence of actions that utilize the environment to change the object's pose. Concretely, we employ hierarchical reinforcement learning to combine a sequence of learned parameterized manipulation primitives. By learning the low-level manipulation policies, our approach can control the object's state through exploiting interactions between the object, the gripper, and the environment. Designing such a complex behavior analytically would be infeasible under uncontrolled conditions, as an analytic approach requires accurate physical modeling of the interaction and contact dynamics. In contrast, we learn a hierarchical policy model that operates directly on depth perception data, without the need for object detection, pose estimation, or manual design of controllers. We evaluate our approach on picking box-shaped objects of various weight, shape, and friction properties from a constrained table-top workspace. Our method transfers to a real robot and is able to successfully complete the object picking task in 98\% of experimental trials. Supplementary information and videos can be found at https://shihminyang.github.io/ED-PMP/.

翻译：许多实际机器人抓取问题中，目标物体的所有抓取姿态均被环境遮挡（例如，被环境本身遮挡）。在这种情况下，单次抓取规划必然失败。因此，必须首先将物体操作至可抓取的构型。我们通过利用环境改变物体姿态的动作序列学习来解决该问题。具体而言，我们采用分层强化学习，将一系列学习的参数化操作基元进行组合。通过构建底层操作策略，我们的方法能够利用物体、夹爪与环境之间的相互作用来调控物体状态。在非受控条件下，设计如此复杂的行为在分析上不可行，因为分析方法需要精确建模相互作用与接触动力学。相反，我们学习一个直接基于深度感知数据的分层策略模型，无需物体检测、姿态估计或手动设计控制器。我们在受限桌面工作空间中评估了对不同重量、形状和摩擦特性的箱形物体的抓取能力。该方法成功迁移至真实机器人，并在98%的实验试次中成功完成物体抓取任务。补充信息与视频详见https://shihminyang.github.io/ED-PMP/。