High-risk applications in robotics, such as robot-assisted surgery, present unique challenges. These systems must be both highly precise and interpretable in order to be deployed in environments with very low tolerance for error or unsafe exploration. We present the first robotic system to demonstrate autonomous clip positioning on a physical phantom in laparoscopic surgery, one of the most common interventions in general surgery. After segmentation of a colorless point cloud from a single camera, target positions for the clips are extracted using spline interpolation, and can then be adjusted by the human operator. The segmentation model is trained on only 60 hand-labeled real point clouds, reflecting data scarcity in the surgical domain. We overcome this with a combination of pre-training on 128,000 synthetic point clouds and two novel data augmentation techniques. The motion of the end-effector to each target is visualized for the operator, satisfying the unique motion constraints of minimally-invasive surgery while ensuring that the robot's actions are verifiable and interpretable. In real robot experiments, our system localizes targets with the required precision of 0.75mm at a 95% success rate and executes autonomous clip positioning with a 100% success rate. We provide insights that are applicable to many other surgical and non-surgical tasks that require identifying and navigating to a precise target. Source code and project page: https://github.com/balazsgyenes/kirurc
翻译:机器人辅助手术等高可靠性应用面临独特挑战。这类系统在容错率极低或禁止危险探索的环境中部署时,必须同时具备高精度与可解释性。本文提出首个在腹腔镜手术物理体模上实现自主施夹的机器人系统——腹腔镜手术是普外科最常见术式之一。通过单摄像头采集无色点云并完成分割后,利用样条插值提取施夹目标位置,操作者可对此进行调整。分割模型仅用60个手工标注的真实点云训练,反映了手术领域的数据稀缺性。我们通过结合128,000个合成点云的预训练与两种新型数据增强技术克服了该局限。系统将末端执行器至各目标的运动轨迹可视化,在满足微创手术特殊运动约束的同时,确保机器人动作可验证可解释。真实机器人实验中,系统以95%成功率实现0.75mm精度定位要求,并以100%成功率完成自主施夹操作。本文提供的见解可推广至众多需要精确目标识别与导航的手术及非手术任务。源代码及项目页面:https://github.com/balazsgyenes/kirurc