Automation holds the potential to assist surgeons in robotic interventions, shifting their mental work load from visuomotor control to high level decision making. Reinforcement learning has shown promising results in learning complex visuomotor policies, especially in simulation environments where many samples can be collected at low cost. A core challenge is learning policies in simulation that can be deployed in the real world, thereby overcoming the sim-to-real gap. In this work, we bridge the visual sim-to-real gap with an image-based reinforcement learning pipeline based on pixel-level domain adaptation and demonstrate its effectiveness on an image-based task in deformable object manipulation. We choose a tissue retraction task because of its importance in clinical reality of precise cancer surgery. After training in simulation on domain-translated images, our policy requires no retraining to perform tissue retraction with a 50% success rate on the real robotic system using raw RGB images. Furthermore, our sim-to-real transfer method makes no assumptions on the task itself and requires no paired images. This work introduces the first successful application of visual sim-to-real transfer for robotic manipulation of deformable objects in the surgical field, which represents a notable step towards the clinical translation of cognitive surgical robotics.

翻译：自动化技术有望在机器人辅助手术中协助外科医生，将其认知负荷从视觉运动控制转向高层决策。强化学习在习得复杂视觉运动策略方面已展现出良好前景，尤其在可低成本采集大量样本的仿真环境中。核心挑战在于学习仿真环境中的策略并使其能部署于现实世界，从而克服仿真到现实的差距。本研究通过基于像素级域适应的图像强化学习流程，弥合了视觉仿真到现实的鸿沟，并在可变形物体操作的图像任务中验证了其有效性。我们选择组织牵拉任务进行研究，因其在精准癌症手术的临床实践中具有重要意义。在经域转换图像仿真训练后，我们的策略无需重新训练即可在真实机器人系统上使用原始RGB图像执行组织牵拉任务，成功率可达50%。此外，我们的仿真到现实迁移方法对任务本身不作任何假设，且无需配对图像。本研究首次成功实现了手术领域可变形物体机器人操作的视觉仿真到现实迁移，标志着认知外科机器人向临床转化迈出了重要一步。