Driven by large-data pre-training, Segment Anything Model (SAM) has been demonstrated as a powerful and promptable framework, revolutionizing the segmentation models. Despite the generality, customizing SAM for specific visual concepts without man-powered prompting is under explored, e.g., automatically segmenting your pet dog in different images. In this paper, we propose a training-free Personalization approach for SAM, termed as PerSAM. Given only a single image with a reference mask, PerSAM first localizes the target concept by a location prior, and segments it within other images or videos via three techniques: target-guided attention, target-semantic prompting, and cascaded post-refinement. In this way, we effectively adapt SAM for private use without any training. To further alleviate the mask ambiguity, we present an efficient one-shot fine-tuning variant, PerSAM-F. Freezing the entire SAM, we introduce two learnable weights for multi-scale masks, only training 2 parameters within 10 seconds for improved performance. To demonstrate our efficacy, we construct a new segmentation dataset, PerSeg, for personalized evaluation, and test our methods on video object segmentation with competitive performance. Besides, our approach can also enhance DreamBooth to personalize Stable Diffusion for text-to-image generation, which discards the background disturbance for better target appearance learning. Code is released at https://github.com/ZrrSkywalker/Personalize-SAM

翻译：受大数据预训练驱动，分割一切模型（Segment Anything Model，SAM）已被证明是一种强大且可提示的框架，彻底改变了分割模型。尽管具有通用性，但针对特定视觉概念进行定制化SAM且无需人工提示的研究仍不充分，例如自动分割不同图像中的宠物狗。本文提出一种无需训练的SAM个性化方法，称为PerSAM。仅需一张带参考掩码的图像，PerSAM首先通过位置先验定位目标概念，并借助三种技术：目标引导注意力、目标语义提示及级联后细化，在其他图像或视频中分割该目标。由此，我们无需任何训练即可有效适配SAM用于个人化用途。为进一步缓解掩码歧义，我们提出一种高效的单样本微调变体PerSAM-F。冻结整个SAM，我们引入两个可学习权重用于多尺度掩码，仅需训练2个参数并在10秒内完成，即可提升性能。为验证有效性，我们构建了一个新的分割数据集PerSeg用于个性化评估，并在视频目标分割任务上测试了所提方法，取得了竞争性性能。此外，我们的方法还可增强DreamBooth以个性化Stable Diffusion进行文本到图像生成，通过去除背景干扰来更好地学习目标外观。代码已开源：https://github.com/ZrrSkywalker/Personalize-SAM