Aiming to predict the complete shapes of partially occluded objects, amodal segmentation is an important step towards visual intelligence. With crucial significance, practical prior knowledge derives from sufficient training, while limited amodal annotations pose challenges to achieve better performance. To tackle this problem, utilizing the mighty priors accumulated in the foundation model, we propose the first SAM-based amodal segmentation approach, PLUG. Methodologically, a novel framework with hierarchical focus is presented to better adapt the task characteristics and unleash the potential capabilities of SAM. In the region level, due to the association and division in visible and occluded areas, inmodal and amodal regions are assigned as the focuses of distinct branches to avoid mutual disturbance. In the point level, we introduce the concept of uncertainty to explicitly assist the model in identifying and focusing on ambiguous points. Guided by the uncertainty map, a computation-economic point loss is applied to improve the accuracy of predicted boundaries. Experiments are conducted on several prominent datasets, and the results show that our proposed method outperforms existing methods with large margins. Even with fewer total parameters, our method still exhibits remarkable advantages.
翻译:模态无关分割旨在预测部分遮挡物体的完整形状,是实现视觉智能的重要步骤。充分的训练能够提供具有关键意义的先验知识,然而有限的模态无关标注数据对性能提升构成了挑战。为解决该问题,我们利用基础模型中积累的强大先验知识,提出了首个基于SAM的模态无关分割方法PLUG。在方法论层面,我们设计了一种具有层级聚焦特性的新颖框架,以更好地适应任务特性并释放SAM的潜在能力。在区域层级,由于可见区域与遮挡区域存在关联与划分,我们将模态内区域与模态无关区域分别设置为不同分支的关注焦点,以避免相互干扰。在点层级,我们引入不确定性概念以显式辅助模型识别并聚焦于模糊点。在不确定性图的引导下,我们采用计算高效的点损失函数来提升预测边界的精度。我们在多个权威数据集上进行了实验,结果表明所提方法以显著优势超越现有方法。即使在总参数量更少的情况下,我们的方法仍展现出显著优势。