Many practically relevant robot grasping problems feature a target object for which all grasps are occluded, e.g., by the environment. Single-shot grasp planning invariably fails in such scenarios. Instead, it is necessary to first manipulate the object into a configuration that affords a grasp. We solve this problem by learning a sequence of actions that utilize the environment to change the object's pose. Concretely, we employ hierarchical reinforcement learning to combine a sequence of learned parameterized manipulation primitives. By learning the low-level manipulation policies, our approach can control the object's state through exploiting interactions between the object, the gripper, and the environment. Designing such a complex behavior analytically would be infeasible under uncontrolled conditions, as an analytic approach requires accurate physical modeling of the interaction and contact dynamics. In contrast, we learn a hierarchical policy model that operates directly on depth perception data, without the need for object detection, pose estimation, or manual design of controllers. We evaluate our approach on picking box-shaped objects of various weight, shape, and friction properties from a constrained table-top workspace. Our method transfers to a real robot and is able to successfully complete the object picking task in 98\% of experimental trials.

翻译：许多实际相关的机器人抓取问题中，目标物体的所有抓取姿态均被遮挡（例如被环境遮挡）。单次抓取规划在这种场景下总是失败。因此，必须先通过操作将物体调整至可抓取的构型。我们通过利用环境改变物体姿态的动作序列学习来解决这一问题。具体而言，我们采用分层强化学习将一系列学习的参数化操作基元进行组合。通过学习底层操作策略，我们的方法能够利用物体、夹爪与环境之间的相互作用来控制物体状态。在非受控条件下，由于接触动力学和交互的精确物理建模难以实现，这种复杂行为的分析设计是不可行的。相比之下，我们学习了一个直接基于深度感知数据运行的分层策略模型，无需物体检测、姿态估计或控制器手动设计。我们在受限桌面工作空间中对不同重量、形状和摩擦特性的箱形物体进行了抓取评估。该方法可迁移至真实机器人，并在98%的实验试次中成功完成物体拾取任务。