The Segment Anything Model (SAM) is the first foundation model for general image segmentation. It designed a novel promotable segmentation task, ensuring zero-shot image segmentation using the pre-trained model via two main modes including automatic everything and manual prompt. SAM has achieved impressive results on various natural image segmentation tasks. However, medical image segmentation (MIS) is more challenging due to the complex modalities, fine anatomical structures, uncertain and complex object boundaries, and wide-range object scales. Meanwhile, zero-shot and efficient MIS can well reduce the annotation time and boost the development of medical image analysis. Hence, SAM seems to be a potential tool and its performance on large medical datasets should be further validated. We collected and sorted 52 open-source datasets, and built a large medical segmentation dataset with 16 modalities, 68 objects, and 553K slices. We conducted a comprehensive analysis of different SAM testing strategies on the so-called COSMOS 553K dataset. Extensive experiments validate that SAM performs better with manual hints like points and boxes for object perception in medical images, leading to better performance in prompt mode compared to everything mode. Additionally, SAM shows remarkable performance in some specific objects and modalities, but is imperfect or even totally fails in other situations. Finally, we analyze the influence of different factors (e.g., the Fourier-based boundary complexity and size of the segmented objects) on SAM's segmentation performance. Extensive experiments validate that SAM's zero-shot segmentation capability is not sufficient to ensure its direct application to the MIS.

翻译：“分割一切模型”（SAM）是首个用于通用图像分割的基础模型。它设计了一种新颖的可提示分割任务，通过两种主要模式（自动全图模式与手动提示模式）确保使用预训练模型实现零样本图像分割。SAM已在多种自然图像分割任务中取得显著成果。然而，医学图像分割因涉及复杂模态、精细解剖结构、不确定且复杂的物体边界以及大范围物体尺度而更具挑战性。同时，零样本且高效的医学图像分割可大幅减少标注时间，推动医学图像分析发展。因此，SAM似乎是一个潜在工具，但其在大型医学数据集上的性能需进一步验证。我们收集整理了52个开源数据集，构建了一个涵盖16种模态、68个目标对象及55.3万张切片的大型医学分割数据集（COSMOS 553K）。针对该数据集，我们全面分析了不同SAM测试策略。大量实验证实：在医学图像目标感知中，SAM结合点、框等手动提示表现更优，提示模式性能优于全图模式。此外，SAM在特定对象和模态中表现卓越，但在其他场景下存在不足甚至完全失效。最后，我们分析了傅里叶边界复杂度、分割目标尺寸等因素对SAM分割性能的影响。大量实验验证表明，SAM的零样本分割能力尚不足以直接应用于医学图像分割。