The Segment Anything Model (SAM) is a powerful foundation model that has revolutionised image segmentation. To apply SAM to surgical instrument segmentation, a common approach is to locate precise points or boxes of instruments and then use them as prompts for SAM in a zero-shot manner. However, we observe two problems with this naive pipeline: (1) the domain gap between natural objects and surgical instruments leads to inferior generalisation of SAM; and (2) SAM relies on precise point or box locations for accurate segmentation, requiring either extensive manual guidance or a well-performing specialist detector for prompt preparation, which leads to a complex multi-stage pipeline. To address these problems, we introduce SurgicalSAM, a novel end-to-end efficient-tuning approach for SAM to effectively integrate surgical-specific information with SAM's pre-trained knowledge for improved generalisation. Specifically, we propose a lightweight prototype-based class prompt encoder for tuning, which directly generates prompt embeddings from class prototypes and eliminates the use of explicit prompts for improved robustness and a simpler pipeline. In addition, to address the low inter-class variance among surgical instrument categories, we propose contrastive prototype learning, further enhancing the discrimination of the class prototypes for more accurate class prompting. The results of extensive experiments on both EndoVis2018 and EndoVis2017 datasets demonstrate that SurgicalSAM achieves state-of-the-art performance while only requiring a small number of tunable parameters. The source code is available at https://github.com/wenxi-yue/SurgicalSAM.

翻译：摘要：Segment Anything Model (SAM) 是一种强大的基础模型，已彻底改变了图像分割领域。为将SAM应用于手术器械分割，常见做法是定位器械的精确点或边界框，然后以零样本方式将其作为SAM的提示。然而，我们观察到这种朴素流程存在两个问题：（1）自然物体与手术器械之间的领域差异导致SAM泛化能力不足；（2）SAM依赖精确的点或框位置实现准确分割，这需要大量人工引导或性能良好的专用检测器来准备提示，从而导致复杂的多阶段流程。针对这些问题，我们提出SurgicalSAM——一种新颖的端到端高效微调方法，旨在将手术专用信息与SAM的预训练知识有效融合，从而提升泛化能力。具体而言，我们设计了一种轻量级基于原型的类别提示编码器进行微调，该编码器直接从类别原型生成提示嵌入，消除了显式提示的使用，从而增强了鲁棒性并简化了流程。此外，为应对手术器械类别间低类间方差的问题，我们提出对比原型学习，进一步增强了类别原型的判别能力，以实现更准确的类别提示。在EndoVis2018和EndoVis2017数据集上的大量实验结果表明，SurgicalSAM在仅需少量可调参数的情况下实现了最先进的性能。源代码已公开于https://github.com/wenxi-yue/SurgicalSAM。