The accurate detection and grasping of transparent objects are challenging but of significance to robots. Here, a visual-tactile fusion framework for transparent object grasping under complex backgrounds and variant light conditions is proposed, including the grasping position detection, tactile calibration, and visual-tactile fusion based classification. First, a multi-scene synthetic grasping dataset generation method with a Gaussian distribution based data annotation is proposed. Besides, a novel grasping network named TGCNN is proposed for grasping position detection, showing good results in both synthetic and real scenes. In tactile calibration, inspired by human grasping, a fully convolutional network based tactile feature extraction method and a central location based adaptive grasping strategy are designed, improving the success rate by 36.7% compared to direct grasping. Furthermore, a visual-tactile fusion method is proposed for transparent objects classification, which improves the classification accuracy by 34%. The proposed framework synergizes the advantages of vision and touch, and greatly improves the grasping efficiency of transparent objects.

翻译：透明物体的精确检测与抓取对机器人而言具有挑战性且意义重大。本文提出了一种适用于复杂背景和变化光照条件下的透明物体抓取视触觉融合框架，包括抓取位置检测、触觉校准以及基于视触觉融合的分类。首先，提出了一种基于高斯分布数据标注的多场景合成抓取数据集生成方法。此外，提出了一种名为TGCNN的新型抓取网络用于抓取位置检测，在合成场景和真实场景中均表现出良好效果。在触觉校准方面，受人类抓取方式启发，设计了一种基于全卷积网络的触觉特征提取方法以及一种基于中心位置的自适应抓取策略，与直接抓取相比，成功率提高了36.7%。进一步提出了一种用于透明物体分类的视触觉融合方法，将分类准确率提升了34%。所提框架协同发挥了视觉与触觉的优势，显著提升了透明物体的抓取效率。