Many practically relevant robot grasping problems feature a target object for which all grasps are occluded, e.g., by the environment. Single-shot grasp planning invariably fails in such scenarios. Instead, it is necessary to first manipulate the object into a configuration that affords a grasp. We solve this problem by learning a sequence of actions that utilize the environment to change the object's pose. Concretely, we employ hierarchical reinforcement learning to combine a sequence of learned parameterized manipulation primitives. By learning the low-level manipulation policies, our approach can control the object's state through exploiting interactions between the object, the gripper, and the environment. Designing such a complex behavior analytically would be infeasible under uncontrolled conditions, as an analytic approach requires accurate physical modeling of the interaction and contact dynamics. In contrast, we learn a hierarchical policy model that operates directly on depth perception data, without the need for object detection, pose estimation, or manual design of controllers. We evaluate our approach on picking box-shaped objects of various weight, shape, and friction properties from a constrained table-top workspace. Our method transfers to a real robot and is able to successfully complete the object picking task in 98\% of experimental trials.

翻译：许多实际相关的机器人抓取问题中，目标物体的所有抓取姿态均被遮挡（例如被环境遮挡）。单次抓取规划在此类场景中必然失败。相反，需要首先将物体操作至可实施抓取的位形。我们通过利用环境改变物体位姿的动作序列学习来解决该问题。具体而言，我们采用分层强化学习来组合一系列学习获得的参数化操作基元。通过学习底层操作策略，我们的方法能够利用物体、夹爪与环境之间的交互作用来控制物体状态。在非受控条件下，分析式设计此类复杂行为是不可行的，因为分析方法需要精确的相互作用与接触动力学物理建模。相比之下，我们学习一个直接作用于深度感知数据的分层策略模型，无需物体检测、姿态估计或控制器的手动设计。我们在受限桌面工作空间中对不同重量、形状和摩擦特性的盒状物体进行拾取任务评估。该方法可迁移至真实机器人，并在98%的实验试次中成功完成物体拾取任务。