This work presents a novel tactile perception-based method, named T-NT, for performing the needle-threading task, an application of deformable linear object (DLO) manipulation. This task is divided into two main stages: Tail-end Finding and Tail-end Insertion. In the first stage, the agent traces the contour of the thread twice using vision-based tactile sensors mounted on the gripper fingers. The two-run tracing is to locate the tail-end of the thread. In the second stage, it employs a tactile-guided reinforcement learning (RL) model to drive the robot to insert the thread into the target needle eyelet. The RL model is trained in a Unity-based simulated environment. The simulation environment supports tactile rendering which can produce realistic tactile images and thread modeling. During insertion, the position of the poke point and the center of the eyelet are obtained through a pre-trained segmentation model, Grounded-SAM, which predicts the masks for both the needle eye and thread imprints. These positions are then fed into the reinforcement learning model, aiding in a smoother transition to real-world applications. Extensive experiments on real robots are conducted to demonstrate the efficacy of our method. More experiments and videos can be found in the supplementary materials and on the website: https://sites.google.com/view/tac-needlethreading.

翻译：本文提出了一种名为T-NT的触觉感知新方法，用于完成柔性线性物体（DLO）操作中的穿针引线任务。该任务分为两个主要阶段：尾端寻找与尾端插入。在第一阶段，机械臂利用安装在夹爪上的视觉触觉传感器对线体轮廓进行两次追踪，通过双程扫描定位线尾端点。在第二阶段，采用触觉引导的强化学习（RL）模型驱动机器人将线体插入目标针眼。该强化学习模型在基于Unity的仿真环境中训练，该环境支持触觉渲染技术，可生成逼真的触觉图像及线体模型。插入过程中，通过预训练的分割模型Grounded-SAM获取戳刺点位置与针眼中心坐标，该模型可同时预测针眼与线体压痕的掩膜。这些位置信息随后输入强化学习模型，有助于实现向真实场景的平滑迁移。通过在真实机器人上开展的大量实验，验证了我们方法的有效性。更多实验内容与演示视频详见补充材料及网站：https://sites.google.com/view/tac-needlethreading。