In this work, we tackle the problem of learning universal robotic dexterous grasping from a point cloud observation under a table-top setting. The goal is to grasp and lift up objects in high-quality and diverse ways and generalize across hundreds of categories and even the unseen. Inspired by successful pipelines used in parallel gripper grasping, we split the task into two stages: 1) grasp proposal (pose) generation and 2) goal-conditioned grasp execution. For the first stage, we propose a novel probabilistic model of grasp pose conditioned on the point cloud observation that factorizes rotation from translation and articulation. Trained on our synthesized large-scale dexterous grasp dataset, this model enables us to sample diverse and high-quality dexterous grasp poses for the object in the point cloud. For the second stage, we propose to replace the motion planning used in parallel gripper grasping with a goal-conditioned grasp policy, due to the complexity involved in dexterous grasping execution. Note that it is very challenging to learn this highly generalizable grasp policy that only takes realistic inputs without oracle states. We thus propose several important innovations, including state canonicalization, object curriculum, and teacher-student distillation. Integrating the two stages, our final pipeline becomes the first to achieve universal generalization for dexterous grasping, demonstrating an average success rate of more than 60% on thousands of object instances, which significantly out performs all baselines, meanwhile showing only a minimal generalization gap.

翻译：本文针对桌面场景下基于点云观测的通用机器人灵巧抓取学习问题展开研究，目标是以高质量、多样化的方式抓取并提起物体，同时实现跨数百个类别甚至未见物体的泛化能力。借鉴平行夹爪抓取的成熟流程，我们将任务分解为两个阶段：1）抓取提议（位姿）生成；2）目标条件抓取执行。第一阶段中，我们提出一种基于点云观测条件建模的新型抓取位姿概率模型，该模型将旋转分量与平移及关节变量进行解耦。基于自建的大规模灵巧抓取数据集训练后，该模型可对点云中物体采样多样化且高质量的灵巧抓取位姿。第二阶段中，鉴于灵巧抓取执行的复杂性，我们提出用目标条件抓取策略替代平行夹爪抓取中使用的运动规划方法。需要指出的是，学习这种仅依赖真实输入（不含理想状态）的高度泛化抓取策略极具挑战性，因此我们提出多项关键创新，包括状态规范化、物体课程学习及师生蒸馏。通过两阶段集成，本框架首次实现灵巧抓取的通用泛化能力，在数千个物体实例上平均成功率超过60%，显著优于所有基线方法，同时展现出极小的泛化差距。