The Segment Anything Model (SAM) has recently gained popularity in the field of image segmentation due to its impressive capabilities in various segmentation tasks and its prompt-based interface. However, recent studies and individual experiments have shown that SAM underperforms in medical image segmentation, since the lack of the medical specific knowledge. This raises the question of how to enhance SAM's segmentation capability for medical images. In this paper, instead of fine-tuning the SAM model, we propose the Medical SAM Adapter (Med-SA), which incorporates domain-specific medical knowledge into the segmentation model using a light yet effective adaptation technique. In Med-SA, we propose Space-Depth Transpose (SD-Trans) to adapt 2D SAM to 3D medical images and Hyper-Prompting Adapter (HyP-Adpt) to achieve prompt-conditioned adaptation. We conduct comprehensive evaluation experiments on 17 medical image segmentation tasks across various image modalities. Med-SA outperforms several state-of-the-art (SOTA) medical image segmentation methods, while updating only 2\% of the parameters. Our code is released at https://github.com/KidsWithTokens/Medical-SAM-Adapter.

翻译：《Segment Anything模型（SAM）因在各类分割任务中展现的卓越能力及基于提示词的交互方式，近期在图像分割领域备受关注。然而，近期研究与独立实验表明，由于缺乏医学领域特定知识，SAM在医学图像分割中表现欠佳。这引发了一个关键问题：如何提升SAM对医学图像的割能力？本文提出了一种名为Medical SAM Adapter（Med-SA）的轻量级高效适配方法，不同于直接微调SAM模型，该方法通过引入领域特异性医学知识来增强分割模型。Med-SA中，我们设计了空间-深度转置（Space-Depth Transpose, SD-Trans）模块将2D SAM适配至3D医学图像，并提出了超提示适配器（Hyper-Prompting Adapter, HyP-Adpt）实现基于提示词的条件化适配。我们在涵盖多种影像模态的17项医学图像分割任务上进行了全面评估实验。结果表明，仅更新2%参数量的Med-SA即可超越多项现有最优（SOTA）医学图像分割方法。相关代码已开源至https://github.com/KidsWithTokens/Medical-SAM-Adapter。