The Segment Anything Model (SAM) has recently gained popularity in the field of image segmentation. Thanks to its impressive capabilities in all-round segmentation tasks and its prompt-based interface, SAM has sparked intensive discussion within the community. It is even said by many prestigious experts that image segmentation task has been "finished" by SAM. However, medical image segmentation, although an important branch of the image segmentation family, seems not to be included in the scope of Segmenting "Anything". Many individual experiments and recent studies have shown that SAM performs subpar in medical image segmentation. A natural question is how to find the missing piece of the puzzle to extend the strong segmentation capability of SAM to medical image segmentation. In this paper, instead of fine-tuning the SAM model, we propose Med SAM Adapter, which integrates the medical specific domain knowledge to the segmentation model, by a simple yet effective adaptation technique. Although this work is still one of a few to transfer the popular NLP technique Adapter to computer vision cases, this simple implementation shows surprisingly good performance on medical image segmentation. A medical image adapted SAM, which we have dubbed Medical SAM Adapter (MSA), shows superior performance on 19 medical image segmentation tasks with various image modalities including CT, MRI, ultrasound image, fundus image, and dermoscopic images. MSA outperforms a wide range of state-of-the-art (SOTA) medical image segmentation methods, such as nnUNet, TransUNet, UNetr, MedSegDiff, and also outperforms the fully fine-turned MedSAM with a considerable performance gap. Code will be released at: https://github.com/WuJunde/Medical-SAM-Adapter.

翻译：Segment Anything Model（SAM）近期在图像分割领域备受关注。凭借其在全向分割任务中的出色能力以及基于提示的交互界面，SAM引发了学术界的广泛讨论。多位权威专家甚至宣称，图像分割任务已被SAM"终结"。然而，医学图像分割作为图像分割领域的重要分支，似乎并未被纳入"任意物体分割"的范畴。大量独立实验与近期研究表明，SAM在医学图像分割中的表现差强人意。一个自然而然的问题是：如何补齐拼图缺失的那一块，将SAM强大的分割能力拓展至医学图像分割？本文中，我们提出Med SAM Adapter（医学SAM适配器），该方案无需微调SAM模型本身，而是通过一种简单却高效的适配技术，将医学领域特定知识整合至分割模型中。尽管本研究仍是少数将流行的NLP技术Adapter迁移至计算机视觉领域的探索之一，但这一简洁实现却在医学图像分割中展现出令人惊喜的效果。医学适应的SAM模型——我们称之为Medical SAM Adapter（MSA）——在涵盖CT、MRI、超声图像、眼底图像和皮肤镜图像等19个医学图像分割任务中表现卓越。MSA不仅优于nnUNet、TransUNet、UNetr、MedSegDiff等众多最先进的医学图像分割方法，更显著超越了完全微调的MedSAM模型，两者之间存在明显的性能差距。代码将发布于：https://github.com/WuJunde/Medical-SAM-Adapter。