We propose a new 6-DoF grasp pose synthesis approach from 2D/2.5D input based on keypoints. Keypoint-based grasp detector from image input has demonstrated promising results in the previous study, where the additional visual information provided by color images compensates for the noisy depth perception. However, it relies heavily on accurately predicting the location of keypoints in the image space. In this paper, we devise a new grasp generation network that reduces the dependency on precise keypoint estimation. Given an RGB-D input, our network estimates both the grasp pose from keypoint detection as well as scale towards the camera. We further re-design the keypoint output space in order to mitigate the negative impact of keypoint prediction noise to Perspective-n-Point (PnP) algorithm. Experiments show that the proposed method outperforms the baseline by a large margin, validating the efficacy of our approach. Finally, despite trained on simple synthetic objects, our method demonstrate sim-to-real capacity by showing competitive results in real-world robot experiments.

翻译：我们提出了一种基于关键点的2D/2.5D输入新型6自由度抓取姿态合成方法。基于关键点的图像输入抓取检测器在先前研究中展现出良好前景，其中彩色图像提供的额外视觉信息可补偿噪声深度感知。然而，该方法严重依赖图像空间关键点位置的精确预测。本文设计了一种新型抓取生成网络，降低了对精确关键点估计的依赖。给定RGB-D输入，网络不仅通过关键点检测估计抓取姿态，还同时估计朝向相机的尺度。我们进一步重新设计了关键点输出空间，以减轻关键点预测噪声对透视n点（PnP）算法的负面影响。实验表明，所提方法大幅优于基线方法，验证了本文方法的有效性。最后，尽管仅在简单合成物体上训练，我们的方法在真实机器人实验中展现了竞争力的结果，证明了从仿真到真实场景的迁移能力。