The Segment Anything Model (SAM) is a recently developed large model for general-purpose segmentation for computer vision tasks. SAM was trained using 11 million images with over 1 billion masks and can produce segmentation results for a wide range of objects in natural scene images. SAM can be viewed as a general perception model for segmentation (partitioning images into semantically meaningful regions). Thus, how to utilize such a large foundation model for medical image segmentation is an emerging research target. This paper shows that although SAM does not immediately give high-quality segmentation for medical image data, its generated masks, features, and stability scores are useful for building and training better medical image segmentation models. In particular, we demonstrate how to use SAM to augment image input for commonly-used medical image segmentation models (e.g., U-Net). Experiments on three segmentation tasks show the effectiveness of our proposed SAMAug method. The code is available at \url{https://github.com/yizhezhang2000/SAMAug}.
翻译:分割一切模型(SAM)是近期开发的面向计算机视觉任务的通用分割大模型。该模型使用1100万张图像及超过10亿个掩码进行训练,能够对自然场景图像中的各类物体生成分割结果。可将其视为通用的分割感知模型(将图像划分为语义上有意义的区域)。因此,如何利用此类大型基础模型进行医学图像分割成为新兴研究目标。本文表明,尽管SAM无法直接对医学图像数据生成高质量分割结果,但其生成的掩码、特征和稳定性分数对于构建和训练更优的医学图像分割模型具有重要价值。我们特别展示了如何利用SAM增强常用医学图像分割模型(如U-Net)的输入图像。在三个分割任务上的实验验证了所提出的SAMAug方法的有效性。代码已开源在 \url{https://github.com/yizhezhang2000/SAMAug}。