The Segment Anything Model (SAM) has recently gained popularity in the field of image segmentation. Thanks to its impressive capabilities in all-round segmentation tasks and its prompt-based interface, SAM has sparked intensive discussion within the community. It is even said by many prestigious experts that image segmentation task has been "finished" by SAM. However, medical image segmentation, although an important branch of the image segmentation family, seems not to be included in the scope of Segmenting "Anything". Many individual experiments and recent studies have shown that SAM performs subpar in medical image segmentation. A natural question is how to find the missing piece of the puzzle to extend the strong segmentation capability of SAM to medical image segmentation. In this paper, instead of fine-tuning the SAM model, we propose Med SAM Adapter, which integrates the medical specific domain knowledge to the segmentation model, by a simple yet effective adaptation technique. Although this work is still one of a few to transfer the popular NLP technique Adapter to computer vision cases, this simple implementation shows surprisingly good performance on medical image segmentation. A medical image adapted SAM, which we have dubbed Medical SAM Adapter (MSA), shows superior performance on 19 medical image segmentation tasks with various image modalities including CT, MRI, ultrasound image, fundus image, and dermoscopic images. MSA outperforms a wide range of state-of-the-art (SOTA) medical image segmentation methods, such as nnUNet, TransUNet, UNetr, MedSegDiff, and also outperforms the fully fine-turned MedSAM with a considerable performance gap. Code will be released at: https://github.com/WuJunde/Medical-SAM-Adapter.

翻译：Segment Anything Model (SAM) 近年来在图像分割领域备受关注。凭借其在全方位分割任务中的卓越能力及基于提示的交互界面，SAM引发了学界的广泛讨论，甚至有多位知名专家声称图像分割任务已被SAM“终结”。然而，作为图像分割分支的重要领域，医学图像分割似乎并未纳入“Segment Anything”的范畴。大量个体实验及近期研究表明，SAM在医学图像分割中表现欠佳。一个自然的问题是如何填补这一缺失环节，将SAM的强分割能力延伸至医学图像分割。本文提出Med SAM Adapter，通过一种简洁而有效的适配技术，将医学领域特定知识融入分割模型，而非对SAM模型进行微调。尽管该工作仍属于将自然语言处理领域流行的Adapter技术迁移至计算机视觉的少数尝试之一，但这种简单的实现方式在医学图像分割中展现出令人惊喜的优异性能。经医学图像适配后的SAM（命名为Medical SAM Adapter, MSA）在包含CT、MRI、超声图像、眼底图像及皮肤镜图像在内的19项医学图像分割任务中均表现出色。MSA不仅优于nnUNet、TransUNet、UNetr、MedSegDiff等众多当前最优（SOTA）医学图像分割方法，且相较于完全微调的MedSAM也具有显著性能优势。代码将发布于：https://github.com/WuJunde/Medical-SAM-Adapter。