Segment Anything Model (SAM), known for its remarkable zero-shot segmentation capabilities, has garnered significant attention in the community. Nevertheless, its performance is challenged when dealing with what we refer to as visually non-salient scenarios, where there is low contrast between the foreground and background. In these cases, existing methods often cannot capture accurate contours and fail to produce promising segmentation results. In this paper, we propose Visually Non-Salient SAM (VNS-SAM), aiming to enhance SAM's perception of visually non-salient scenarios while preserving its original zero-shot generalizability. We achieve this by effectively exploiting SAM's low-level features through two designs: Mask-Edge Token Interactive decoder and Non-Salient Feature Mining module. These designs help the SAM decoder gain a deeper understanding of non-salient characteristics with only marginal parameter increments and computational requirements. The additional parameters of VNS-SAM can be optimized within 4 hours, demonstrating its feasibility and practicality. In terms of data, we established VNS-SEG, a unified dataset for various VNS scenarios, with more than 35K images, in contrast to previous single-task adaptations. It is designed to make the model learn more robust VNS features and comprehensively benchmark the model's segmentation performance and generalizability on VNS scenarios. Extensive experiments across various VNS segmentation tasks demonstrate the superior performance of VNS-SAM, particularly under zero-shot settings, highlighting its potential for broad real-world applications. Codes and datasets are publicly available at https://guangqian-guo.github.io/VNS-SAM.

翻译：Segment Anything Model (SAM)以其卓越的零样本分割能力在学界引起了广泛关注。然而，在处理我们称之为视觉非显著场景时，其性能面临挑战。这类场景中前景与背景对比度较低，现有方法通常难以捕捉精确轮廓，也无法产生理想的分割结果。本文提出视觉非显著SAM（VNS-SAM），旨在增强SAM对视觉非显著场景的感知能力，同时保持其原有的零样本泛化性。我们通过两种设计有效利用SAM的低层特征：掩码-边缘令牌交互解码器与非显著特征挖掘模块。这些设计使SAM解码器能以极少的参数增量和计算需求，更深入地理解非显著特征。VNS-SAM的额外参数可在4小时内完成优化，证明了其可行性与实用性。在数据方面，我们构建了VNS-SEG——一个涵盖多种VNS场景的统一数据集，包含超过3.5万张图像，与以往的单任务适配方法形成对比。该数据集旨在让模型学习更鲁棒的VNS特征，并全面评估模型在VNS场景下的分割性能与泛化能力。在多种VNS分割任务上的大量实验表明，VNS-SAM具有优越性能，尤其在零样本设置下，凸显了其在广泛实际应用中的潜力。代码与数据集已公开于https://guangqian-guo.github.io/VNS-SAM。