In this paper, we examine the recent Segment Anything Model (SAM) on medical images, and report both quantitative and qualitative zero-shot segmentation results on nine medical image segmentation benchmarks, covering various imaging modalities, such as optical coherence tomography (OCT), magnetic resonance imaging (MRI), and computed tomography (CT), as well as different applications including dermatology, ophthalmology, and radiology. Those benchmarks are representative and commonly used in model development. Our experimental results indicate that while SAM presents remarkable segmentation performance on images from the general domain, its zero-shot segmentation ability remains restricted for out-of-distribution images, e.g., medical images. In addition, SAM exhibits inconsistent zero-shot segmentation performance across different unseen medical domains. For certain structured targets, e.g., blood vessels, the zero-shot segmentation of SAM completely failed. In contrast, a simple fine-tuning of it with a small amount of data could lead to remarkable improvement of the segmentation quality, showing the great potential and feasibility of using fine-tuned SAM to achieve accurate medical image segmentation for a precision diagnostics. Our study indicates the versatility of generalist vision foundation models on medical imaging, and their great potential to achieve desired performance through fine-turning and eventually address the challenges associated with accessing large and diverse medical datasets in support of clinical diagnostics.

翻译：本文研究了最新提出的Segment Anything模型（SAM）在医学图像上的表现，报告了其在九个医学图像分割基准上的定量与定性零样本分割结果。这些基准覆盖多种成像模态，如光学相干断层扫描（OCT）、磁共振成像（MRI）和计算机断层扫描（CT），以及皮肤科、眼科和放射科等不同应用领域。这些基准具有代表性且广泛应用于模型开发。实验结果表明，尽管SAM在通用领域图像上表现出卓越的分割性能，但其在分布外图像（如医学图像）上的零样本分割能力仍受限。此外，SAM在不同未见医学领域间的零样本分割性能存在不一致性。对于某些结构化目标（如血管），SAM的零样本分割完全失败。相比之下，使用少量数据对SAM进行简单微调可显著提升分割质量，表明通过微调SAM实现精准医学图像分割用于精确诊断的巨大潜力和可行性。本研究揭示了通用视觉基础模型在医学影像中的多功能性，以及其通过微调达到理想性能、最终解决获取大规模多样化医学数据集以支持临床诊断所面临挑战的巨大潜力。