In this work, we present SuFIA, the first framework for natural language-guided augmented dexterity for robotic surgical assistants. SuFIA incorporates the strong reasoning capabilities of large language models (LLMs) with perception modules to implement high-level planning and low-level control of a robot for surgical sub-task execution. This enables a learning-free approach to surgical augmented dexterity without any in-context examples or motion primitives. SuFIA uses a human-in-the-loop paradigm by restoring control to the surgeon in the case of insufficient information, mitigating unexpected errors for mission-critical tasks. We evaluate SuFIA on four surgical sub-tasks in a simulation environment and two sub-tasks on a physical surgical robotic platform in the lab, demonstrating its ability to perform common surgical sub-tasks through supervised autonomous operation under challenging physical and workspace conditions. Project website: orbit-surgical.github.io/sufia

翻译：本文提出SuFIA，这是首个面向机器人手术辅助自然语言引导增强灵巧操作的框架。SuFIA融合大型语言模型的强大推理能力与感知模块，实现机器人执行手术子任务的高层级规划与低层级控制。该方案无需学习过程、无需上下文示例或运动基元即可完成手术增强灵巧操作。SuFIA采用人在回路范式，当信息不足时将控制权交还给外科医生，从而降低关键任务中的意外错误风险。我们在仿真环境中对四个手术子任务进行了评估，并在实验室物理手术机器人平台上完成了两个子任务的测试，结果表明该框架能在具有挑战性的物理环境与工作空间条件下，通过受监督自主操作执行常见手术子任务。项目网站：orbit-surgical.github.io/sufia