In this paper, we explore a principal way to enhance the quality of widely pre-existing coarse masks, enabling them to serve as reliable training data for segmentation models to reduce the annotation cost. In contrast to prior refinement techniques that are tailored to specific models or tasks in a close-world manner, we propose SAMRefiner, a universal and efficient approach by adapting SAM to the mask refinement task. The core technique of our model is the noise-tolerant prompting scheme. Specifically, we introduce a multi-prompt excavation strategy to mine diverse input prompts for SAM (i.e., distance-guided points, context-aware elastic bounding boxes, and Gaussian-style masks) from initial coarse masks. These prompts can collaborate with each other to mitigate the effect of defects in coarse masks. In particular, considering the difficulty of SAM to handle the multi-object case in semantic segmentation, we introduce a split-then-merge (STM) pipeline. Additionally, we extend our method to SAMRefiner++ by introducing an additional IoU adaption step to further boost the performance of the generic SAMRefiner on the target dataset. This step is self-boosted and requires no additional annotation. The proposed framework is versatile and can flexibly cooperate with existing segmentation methods. We evaluate our mask framework on a wide range of benchmarks under different settings, demonstrating better accuracy and efficiency. SAMRefiner holds significant potential to expedite the evolution of refinement tools. Our code is available at https://github.com/linyq2117/SAMRefiner.

翻译：本文探索了一种提升现有广泛存在的粗糙掩码质量的根本方法，使其能够作为分割模型的可靠训练数据以降低标注成本。与以往针对特定模型或任务、以封闭世界方式设计的精细化技术不同，我们提出SAMRefiner，一种通过将SAM适配到掩码精细化任务的通用且高效的方法。我们模型的核心技术是噪声容忍的提示方案。具体而言，我们引入了一种多提示挖掘策略，从初始粗糙掩码中为SAM挖掘多样化的输入提示（即距离引导点、上下文感知弹性边界框和高斯风格掩码）。这些提示可以相互协作，以减轻粗糙掩码中缺陷的影响。特别地，考虑到SAM处理语义分割中多目标情况的困难，我们引入了一个先分割后合并（STM）的流程。此外，我们通过引入一个额外的IoU适配步骤，将我们的方法扩展到SAMRefiner++，以进一步提升通用SAMRefiner在目标数据集上的性能。该步骤是自增强的，无需额外标注。所提出的框架具有通用性，可以灵活地与现有的分割方法协作。我们在不同设置下的广泛基准测试中评估了我们的掩码框架，证明了其更好的准确性和效率。SAMRefiner在加速精细化工具的发展方面具有巨大潜力。我们的代码可在https://github.com/linyq2117/SAMRefiner获取。