Recent advancements in retinal surgery have paved the way for a modern operating room equipped with a surgical robot, a microscope, and intraoperative optical coherence tomography (iOCT)- a depth sensor widely used in retinal surgery. Integrating these tools raises the fundamental question of how to effectively combine them to enable surgical autonomy. In this work, we tackle this question by developing a unified framework that facilitates real-time autonomous surgical workflows leveraging these devices. The system features: (1) a novel imaging system that integrates the microscope and iOCT in real-time by dynamically tracking the surgical instrument via a small iOCT scanning region, providing real-time depth feedback; (2) implementation of convolutional neural networks (CNN) that automatically detect and segment task-relevant information for surgical autonomy; (3) intuitive selection of goal waypoints within both the microscope and iOCT views through simple mouse-click interactions; and (4) integration of model predictive control (MPC) for trajectory generation, ensuring patient safety by implementing safety-related kinematic constraints. The system's utility is demonstrated by automating subretinal injection (SI), a challenging procedure with high accuracy and depth perception requirements. We validate our system by conducting 30 successful SI trials on pig eyes, achieving mean needle insertion accuracy of 26 micrometers to various subretinal goals and mean duration of 55 seconds. Preliminary comparisons to a human operator performing SI in robot-assisted mode highlight the enhanced safety of our system. Project website is here: https://sites.google.com/view/eyesurgerymicroscopeoct/home

翻译：视网膜手术的最新进展为现代手术室配备了手术机器人、显微镜及术中光学相干断层扫描（iOCT）——一种在视网膜手术中广泛使用的深度传感器。这些工具的集成提出了一个基本问题：如何有效组合它们以实现手术自主性？在本研究中，我们通过开发一个统一框架来应对这一问题，该框架利用这些设备实现实时自主手术流程。系统特点包括：（1）一种新颖的成像系统，通过动态跟踪手术器械（利用小型iOCT扫描区域）实时集成显微镜与iOCT，提供实时深度反馈；（2）实现卷积神经网络（CNN），自动检测并分割任务相关信息以支持手术自主性；（3）通过简单的鼠标点击交互，在显微镜和iOCT视图中直观选择目标路径点；（4）集成模型预测控制（MPC）进行轨迹生成，并通过实施安全相关运动学约束确保患者安全。该系统通过自动化视网膜下注射（SI）——一项对精度和深度感知要求极高的具有挑战性的手术——展示了其实用性。我们在猪眼上进行30次成功SI试验验证系统，实现了平均针尖插入精度26微米（针对不同视网膜下目标），平均操作时长55秒。与操作员在机器人辅助模式下进行SI的初步对比凸显了我们系统更强的安全性。项目网站见：https://sites.google.com/view/eyesurgerymicroscopeoct/home