The Segment Anything Model (SAM) is a recently proposed prompt-based segmentation model in a generic zero-shot segmentation approach. With the zero-shot segmentation capacity, SAM achieved impressive flexibility and precision on various segmentation tasks. However, the current pipeline requires manual prompts during the inference stage, which is still resource intensive for biomedical image segmentation. In this paper, instead of using prompts during the inference stage, we introduce a pipeline that utilizes the SAM, called all-in-SAM, through the entire AI development workflow (from annotation generation to model finetuning) without requiring manual prompts during the inference stage. Specifically, SAM is first employed to generate pixel-level annotations from weak prompts (e.g., points, bounding box). Then, the pixel-level annotations are used to finetune the SAM segmentation model rather than training from scratch. Our experimental results reveal two key findings: 1) the proposed pipeline surpasses the state-of-the-art (SOTA) methods in a nuclei segmentation task on the public Monuseg dataset, and 2) the utilization of weak and few annotations for SAM finetuning achieves competitive performance compared to using strong pixel-wise annotated data.

翻译：分割一切模型（SAM）是近期提出的基于提示的通用零样本分割模型。凭借零样本分割能力，SAM在多种分割任务中展现了出色的灵活性与精度。然而，当前流程在推理阶段仍需手动提示，这对生物医学图像分割而言仍属资源密集型操作。本文提出一种名为All-in-SAM的完整流水线，在整个AI开发工作流中（从标注生成到模型微调）无需在推理阶段使用手动提示即可利用SAM。具体而言，首先利用SAM从弱提示（如点、边界框）生成像素级标注，随后使用这些像素级标注微调SAM分割模型，而非从头训练。实验结果表明两个关键发现：1）在公开Monuseg数据集上的细胞核分割任务中，所提流水线超越当前最优方法；2）使用少量弱标注进行SAM微调，其性能可与基于强像素级标注数据的训练结果相抗衡。