Training segmentation models for medical images continues to be challenging due to the limited availability of data annotations. Segment Anything Model (SAM) is a foundation model that is intended to segment user-defined objects of interest in an interactive manner. While the performance on natural images is impressive, medical image domains pose their own set of challenges. Here, we perform an extensive evaluation of SAM's ability to segment medical images on a collection of 19 medical imaging datasets from various modalities and anatomies. We report the following findings: (1) SAM's performance based on single prompts highly varies depending on the dataset and the task, from IoU=0.1135 for spine MRI to IoU=0.8650 for hip X-ray. (2) Segmentation performance appears to be better for well-circumscribed objects with prompts with less ambiguity and poorer in various other scenarios such as the segmentation of brain tumors. (3) SAM performs notably better with box prompts than with point prompts. (4) SAM outperforms similar methods RITM, SimpleClick, and FocalClick in almost all single-point prompt settings. (5) When multiple-point prompts are provided iteratively, SAM's performance generally improves only slightly while other methods' performance improves to the level that surpasses SAM's point-based performance. We also provide several illustrations for SAM's performance on all tested datasets, iterative segmentation, and SAM's behavior given prompt ambiguity. We conclude that SAM shows impressive zero-shot segmentation performance for certain medical imaging datasets, but moderate to poor performance for others. SAM has the potential to make a significant impact in automated medical image segmentation in medical imaging, but appropriate care needs to be applied when using it.

翻译：由于数据标注的有限性，训练医学图像的分割模型仍然具有挑战性。Segment Anything Model（SAM）是一个基础模型，旨在以交互方式分割用户定义的目标对象。尽管其在自然图像上的表现令人印象深刻，但医学图像领域带来了其自身的挑战。在此，我们对SAM在医学图像分割方面的能力进行了广泛评估，使用了来自不同模态和解剖结构的19个医学影像数据集。我们报告了以下发现：（1）基于单次提示的SAM性能因数据集和任务而异，从脊柱MRI的IoU=0.1135到髋部X射线的IoU=0.8650。（2）对于边界清晰的目标，在歧义性较少的提示下，分割性能更好，而在其他各种场景（如脑肿瘤分割）中表现较差。（3）SAM在使用框提示时明显优于点提示。（4）在几乎所有单点提示设置中，SAM优于类似方法RITM、SimpleClick和FocalClick。（5）当迭代提供多点提示时，SAM的性能通常仅略有提升，而其他方法的性能提升至超越SAM基于点性能的水平。我们还提供了SAM在所有测试数据集上的性能、迭代分割以及提示歧义行为的多项图示。我们得出结论：SAM在部分医学影像数据集上表现出令人瞩目的零样本分割性能，但在其他数据集上表现中等至较差。SAM在自动医学图像分割领域具有产生重大影响的潜力，但在使用时需谨慎。