Needle picking is a challenging manipulation task in robot-assisted surgery due to the characteristics of small slender shapes of needles, needles' variations in shapes and sizes, and demands for millimeter-level control. Prior works, heavily relying on the prior of needles (e.g., geometric models), are hard to scale to unseen needles' variations. In this paper, we present the first end-to-end learning method to train deep visuomotor policy for needle picking. Concretely, we propose DreamerfD to maximally leverage demonstrations to improve the learning efficiency of a state-of-the-art model-based reinforcement learning method, DreamerV2; Since Variational Auto-Encoder (VAE) in DreamerV2 is difficult to scale to high-resolution images, we propose Dynamic Spotlight Adaptation to represent control-related visual signals in a low-resolution image space; Virtual Clutch is also proposed to reduce performance degradation due to significant error between prior and posterior encoded states at the beginning of a rollout. We conducted extensive experiments in simulation to evaluate the performance, robustness, in-domain variation adaptation, and effectiveness of individual components of our method. Our method, trained by 8k demonstration timesteps and 140k online policy timesteps, can achieve a remarkable success rate of 80%. Furthermore, our method effectively demonstrated its superiority in generalization to unseen in-domain variations including needle variations and image disturbance, highlighting its robustness and versatility. Codes and videos are available at https://sites.google.com/view/DreamerfD.

翻译：针抓取是机器人辅助手术中的一项具有挑战性的操作任务，其难点在于针体细长、形状尺寸各异，且需要毫米级精度控制。现有方法高度依赖针的几何模型等先验信息，难以泛化至未见过的针型变体。本文首次提出端到端学习方法，用于训练针抓取任务的深度视觉运动策略。具体而言，我们提出DreamerfD方法，最大程度利用示教数据提升基于模型的最优强化学习方法DreamerV2的学习效率；针对DreamerV2中变分自编码器难以处理高分辨率图像的问题，我们提出动态焦点适配技术，将控制相关视觉信号映射至低分辨率图像空间；同时引入虚拟离合器机制，以减少轨迹初始阶段编码状态先验与后验间显著差异导致的性能衰退。我们在仿真环境中开展大量实验，评估了方法的性能、鲁棒性、域内变体适应能力及各组件的有效性。本方法仅需8千步示教数据和14万步在线策略学习即可实现80%的显著成功率。此外，方法在针型变化和图像干扰等未见过域内变体泛化测试中展现出卓越优势，验证了其鲁棒性与通用性。代码与视频详见https://sites.google.com/view/DreamerfD。