The Segment Anything Model (SAM) has recently gained popularity in the field of image segmentation. Thanks to its impressive capabilities in all-round segmentation tasks and its prompt-based interface, SAM has sparked intensive discussion within the community. It is even said by many prestigious experts that image segmentation task has been "finished" by SAM. However, medical image segmentation, although an important branch of the image segmentation family, seems not to be included in the scope of Segmenting "Anything". Many individual experiments and recent studies have shown that SAM performs subpar in medical image segmentation. A natural question is how to find the missing piece of the puzzle to extend the strong segmentation capability of SAM to medical image segmentation. In this paper, instead of fine-tuning the SAM model, we propose Med SAM Adapter, which integrates the medical specific domain knowledge to the segmentation model, by a simple yet effective adaptation technique. Although this work is still one of a few to transfer the popular NLP technique Adapter to computer vision cases, this simple implementation shows surprisingly good performance on medical image segmentation. A medical image adapted SAM, which we have dubbed Medical SAM Adapter (MSA), shows superior performance on 19 medical image segmentation tasks with various image modalities including CT, MRI, ultrasound image, fundus image, and dermoscopic images. MSA outperforms a wide range of state-of-the-art (SOTA) medical image segmentation methods, such as nnUNet, TransUNet, UNetr, MedSegDiff, and also outperforms the fully fine-turned MedSAM with a considerable performance gap. Code will be released at: https://github.com/WuJunde/Medical-SAM-Adapter.

翻译：分割一切模型（Segment Anything Model, SAM）近期在图像分割领域备受关注。凭借其在全方位分割任务中的卓越能力及基于提示的交互界面，SAM引发了学界的广泛讨论。多位权威专家甚至宣称，图像分割任务已被SAM"终结"。然而，作为图像分割体系的重要分支，医学图像分割似乎未被纳入"分割一切"的范畴。大量独立实验与近期研究表明，SAM在医学图像分割中的表现不尽如人意。一个自然的问题是：如何找到缺失的拼图，将SAM强大的分割能力拓展至医学图像分割领域？本文提出Med SAM适配器（Med SAM Adapter），不同于微调SAM模型，该方法通过简洁而有效的适配技术将医学领域特定知识融入分割模型。尽管本研究仍属少数将NLP领域的适配器技术迁移至计算机视觉的探索，但这一简洁实现却在医学图像分割中展现出惊人的性能。经医学图像适配的SAM（我们将其命名为Medical SAM Adapter, MSA）在涵盖CT、MRI、超声图像、眼底图像及皮肤镜图像等19种不同模态的医学图像分割任务中表现优越。MSA不仅全面超越nnUNet、TransUNet、UNetr、MedSegDiff等主流医学图像分割方法，更以显著性能差距优于完全微调的MedSAM。代码将发布在：https://github.com/WuJunde/Medical-SAM-Adapter。