The emergence of large models, also known as foundation models, has brought significant advancements to AI research. One such model is Segment Anything (SAM), which is designed for image segmentation tasks. However, as with other foundation models, our experimental findings suggest that SAM may fail or perform poorly in certain segmentation tasks, such as shadow detection and camouflaged object detection (concealed object detection). This study first paves the way for applying the large pre-trained image segmentation model SAM to these downstream tasks, even in situations where SAM performs poorly. Rather than fine-tuning the SAM network, we propose \textbf{SAM-Adapter}, which incorporates domain-specific information or visual prompts into the segmentation network by using simple yet effective adapters. Our extensive experiments show that SAM-Adapter can significantly elevate the performance of SAM in challenging tasks and we can even outperform task-specific network models and achieve state-of-the-art performance in the task we tested: camouflaged object detection and shadow detection. We believe our work opens up opportunities for utilizing SAM in downstream tasks, with potential applications in various fields, including medical image processing, agriculture, remote sensing, and more.

翻译：大模型（即基础模型）的出现为人工智能研究带来了重大进展。其中，Segment Anything（SAM）模型专为图像分割任务设计。然而，与其他基础模型类似，我们的实验发现SAM在某些分割任务中可能失败或表现欠佳，例如阴影检测和伪装目标检测（隐蔽目标检测）。本工作首先为将大型预训练图像分割模型SAM应用于这些下游任务铺平了道路，即便在SAM表现欠佳的场景中亦是如此。不同于对SAM网络进行微调，我们提出**SAM-Adapter**，通过使用简单而有效的适配器将领域特定信息或视觉提示融入分割网络。大量实验表明，SAM-Adapter能够显著提升SAM在挑战性任务中的性能，甚至超越任务特定网络模型，在我们测试的伪装目标检测和阴影检测任务中达到最先进水平。我们相信，本工作为SAM在下游任务中的应用开辟了机遇，并有望应用于医学图像处理、农业、遥感等多个领域。