The performance of image segmentation models has historically been constrained by the high cost of collecting large-scale annotated data. The Segment Anything Model (SAM) alleviates this original problem through a promptable, semantics-agnostic, segmentation paradigm and yet still requires manual visual-prompts or complex domain-dependent prompt-generation rules to process a new image. Towards reducing this new burden, our work investigates the task of object segmentation when provided with, alternatively, only a small set of reference images. Our key insight is to leverage strong semantic priors, as learned by foundation models, to identify corresponding regions between a reference and a target image. We find that correspondences enable automatic generation of instance-level segmentation masks for downstream tasks and instantiate our ideas via a multi-stage, training-free method incorporating (1) memory bank construction; (2) representation aggregation and (3) semantic-aware feature matching. Our experiments show significant improvements on segmentation metrics, leading to state-of-the-art performance on COCO FSOD (36.8% nAP), PASCAL VOC Few-Shot (71.2% nAP50) and outperforming existing training-free approaches on the Cross-Domain FSOD benchmark (22.4% nAP).

翻译：图像分割模型的性能历来受限于大规模标注数据的高昂收集成本。Segment Anything Model（SAM）通过一种可提示、语义无关的分割范式缓解了这一原始问题，但在处理新图像时仍需手动视觉提示或复杂的领域相关提示生成规则。为减轻这一新负担，本研究探讨了在仅提供少量参考图像情况下的物体分割任务。我们的核心洞见在于利用基础模型所学习的强语义先验，识别参考图像与目标图像之间的对应区域。研究发现，这种对应关系能够为下游任务自动生成实例级分割掩码，并通过一个多阶段、免训练的方法实现了这一理念，该方法包含：（1）记忆库构建；（2）表征聚合与（3）语义感知特征匹配。实验结果表明，该方法在分割指标上取得显著提升，在COCO FSOD（36.8% nAP）、PASCAL VOC Few-Shot（71.2% nAP50）上达到最先进性能，并在跨领域FSOD基准测试（22.4% nAP）上优于现有免训练方法。