In semantic segmentation, accurate prediction masks are crucial for downstream tasks such as medical image analysis and image editing. Due to the lack of annotated data, few-shot semantic segmentation (FSS) performs poorly in predicting masks with precise contours. Recently, we have noticed that the large foundation model segment anything model (SAM) performs well in processing detailed features. Inspired by SAM, we propose FSS-SAM to boost FSS methods by addressing the issue of inaccurate contour. The FSS-SAM is training-free. It works as a post-processing tool for any FSS methods and can improve the accuracy of predicted masks. Specifically, we use predicted masks from FSS methods to generate prompts and then use SAM to predict new masks. To avoid predicting wrong masks with SAM, we propose a prediction result selection (PRS) algorithm. The algorithm can remarkably decrease wrong predictions. Experiment results on public datasets show that our method is superior to base FSS methods in both quantitative and qualitative aspects.
翻译:在语义分割任务中,准确的预测掩膜对于医学图像分析、图像编辑等下游任务至关重要。由于标注数据匮乏,少样本语义分割(FSS)在预测具有精确轮廓的掩膜时表现不佳。近期我们发现,大型基础模型Segment Anything Model(SAM)在处理细节特征方面表现出色。受SAM启发,我们提出FSS-SAM方法,通过解决轮廓不精确的问题来增强FSS方法性能。FSS-SAM无需训练,可作为任意FSS方法的后处理工具,提升预测掩膜的准确率。具体而言,我们利用FSS方法生成的预测掩膜构建提示,再通过SAM预测新掩膜。为避免SAM产生错误预测,我们提出预测结果选择(PRS)算法,该算法能显著降低错误预测率。在公开数据集上的实验结果表明,本方法在定量和定性指标上均优于基础FSS方法。