The Segment Anything Model (SAM) has recently gained popularity in the field of image segmentation. Thanks to its impressive capabilities in all-round segmentation tasks and its prompt-based interface, SAM has sparked intensive discussion within the community. It is even said by many prestigious experts that image segmentation task has been "finished" by SAM. However, medical image segmentation, although an important branch of the image segmentation family, seems not to be included in the scope of Segmenting "Anything". Many individual experiments and recent studies have shown that SAM performs subpar in medical image segmentation. A natural question is how to find the missing piece of the puzzle to extend the strong segmentation capability of SAM to medical image segmentation. In this paper, instead of fine-tuning the SAM model, we propose Med SAM Adapter, which integrates the medical specific domain knowledge to the segmentation model, by a simple yet effective adaptation technique. Although this work is still one of a few to transfer the popular NLP technique Adapter to computer vision cases, this simple implementation shows surprisingly good performance on medical image segmentation. A medical image adapted SAM, which we have dubbed Medical SAM Adapter (MSA), shows superior performance on 19 medical image segmentation tasks with various image modalities including CT, MRI, ultrasound image, fundus image, and dermoscopic images. MSA outperforms a wide range of state-of-the-art (SOTA) medical image segmentation methods, such as nnUNet, TransUNet, UNetr, MedSegDiff, and also outperforms the fully fine-turned MedSAM with a considerable performance gap. Code will be released at: https://github.com/WuJunde/Medical-SAM-Adapter.

翻译：Segment Anything模型（SAM）近期在图像分割领域广受关注。凭借其在通用分割任务中的卓越能力以及基于提示的交互界面，SAM引发了学界的广泛讨论，甚至多位权威专家宣称图像分割任务已被SAM "终结"。然而，医学图像分割作为图像分割领域的重要分支，似乎并未被纳入"分割万物"的范畴。多项独立实验与近期研究表明，SAM在医学图像分割中的表现欠佳。自然产生的问题是：如何补全这一拼图缺失的环节，将SAM强大的分割能力延伸至医学图像分割？本文提出Med SAM Adapter，该方法并非对SAM模型进行微调，而是通过简单而有效的适配技术，将医学领域知识融入分割模型。尽管这项研究仍属少数将NLP领域的Adapter技术迁移至计算机视觉任务的尝试，但这一简洁实现却在医学图像分割中展现出令人惊喜的性能表现。我们称为医学SAM适配器（MSA）的医学图像适配版本，在包含CT、MRI、超声图像、眼底图像及皮肤镜图像等19项医学图像分割任务中表现卓越。MSA不仅优于nnUNet、TransUNet、UNetr、MedSegDiff等众多最先进（SOTA）的医学图像分割方法，更以显著性能差距超越了完全微调的MedSAM。代码将发布于：https://github.com/WuJunde/Medical-SAM-Adapter。