Segment anything model (SAM) has shown its spectacular performance in segmenting universal objects, especially when elaborate prompts are provided. However, the drawback of SAM is twofold. On the first hand, it fails to segment specific targets, e.g., shadow images or lesions in medical images. On the other hand, manually specifying prompts is extremely time-consuming. To overcome the problems, we propose AdapterShadow, which adapts SAM model for shadow detection. To adapt SAM for shadow images, trainable adapters are inserted into the frozen image encoder of SAM, since the training of the full SAM model is both time and memory consuming. Moreover, we introduce a novel grid sampling method to generate dense point prompts, which helps to automatically segment shadows without any manual interventions. Extensive experiments are conducted on four widely used benchmark datasets to demonstrate the superior performance of our proposed method. Codes will are publicly available at https://github.com/LeipingJie/AdapterShadow.
翻译:分割一切模型(SAM)在处理通用物体分割时展现了卓越性能,尤其在提供精细提示的情况下。然而,SAM存在两方面缺陷:一方面,它无法分割特定目标,例如阴影图像或医学图像中的病灶;另一方面,手动指定提示极为耗时。为克服这些问题,我们提出AdapterShadow,通过适配SAM模型实现阴影检测。由于完整SAM模型的训练既耗时又耗内存,我们在SAM的冻结图像编码器中插入可训练适配器,使其适配阴影图像。此外,我们引入一种新颖的网格采样方法生成密集点提示,从而无需任何人工干预即可自动分割阴影。在四个广泛使用的基准数据集上进行了大量实验,证明了所提方法的优越性能。代码将开源在 https://github.com/LeipingJie/AdapterShadow。