The Segment Anything Model (SAM) is the first foundation model for general image segmentation. It designed a novel promotable segmentation task, ensuring zero-shot image segmentation using the pre-trained model via two main modes including automatic everything and manual prompt. SAM has achieved impressive results on various natural image segmentation tasks. However, medical image segmentation (MIS) is more challenging due to the complex modalities, fine anatomical structures, uncertain and complex object boundaries, and wide-range object scales. Meanwhile, zero-shot and efficient MIS can well reduce the annotation time and boost the development of medical image analysis. Hence, SAM seems to be a potential tool and its performance on large medical datasets should be further validated. We collected and sorted 52 open-source datasets, and built a large medical segmentation dataset with 16 modalities, 68 objects, and 553K slices. We conducted a comprehensive analysis of different SAM testing strategies on the so-called COSMOS 553K dataset. Extensive experiments validate that SAM performs better with manual hints like points and boxes for object perception in medical images, leading to better performance in prompt mode compared to everything mode. Additionally, SAM shows remarkable performance in some specific objects and modalities, but is imperfect or even totally fails in other situations. Finally, we analyze the influence of different factors (e.g., the Fourier-based boundary complexity and size of the segmented objects) on SAM's segmentation performance. Extensive experiments validate that SAM's zero-shot segmentation capability is not sufficient to ensure its direct application to the MIS.

翻译：分割一切模型（SAM）是首个面向通用图像分割的基础模型。它设计了一种新颖的可提示分割任务，通过自动全图分割和手动提示两种模式，利用预训练模型实现零样本图像分割。SAM在多种自然图像分割任务中取得了显著成果。然而，医学图像分割因模态复杂、解剖结构精细、目标边界不确定且复杂、目标尺度差异大而更具挑战性。同时，零样本且高效的医学图像分割能够有效减少标注时间，推动医学图像分析的发展。因此，SAM被视为一种潜在工具，其在大型医学数据集上的性能有待进一步验证。我们收集整理了52个开源数据集，构建了涵盖16种模态、68个目标对象、55.3万张切片的大型医学分割数据集（COSMOS 553K）。针对该数据集，我们对SAM的不同测试策略进行了全面分析。大量实验验证表明，SAM在医学图像目标感知中采用点、框等手动提示时表现更优，提示模式性能优于全图模式。此外，SAM在某些特定目标和模态中表现卓越，但在其他情况下存在缺陷甚至完全失效。最后，我们分析了傅里叶边界复杂度、分割对象尺寸等因素对SAM分割性能的影响。大量实验证实，SAM的零样本分割能力尚不足以直接应用于医学图像分割领域。