Driven by large-data pre-training, Segment Anything Model (SAM) has been demonstrated as a powerful and promptable framework, revolutionizing the segmentation models. Despite the generality, customizing SAM for specific visual concepts without man-powered prompting is under explored, e.g., automatically segmenting your pet dog in different images. In this paper, we propose a training-free Personalization approach for SAM, termed as PerSAM. Given only a single image with a reference mask, PerSAM first localizes the target concept by a location prior, and segments it within other images or videos via three techniques: target-guided attention, target-semantic prompting, and cascaded post-refinement. In this way, we effectively adapt SAM for private use without any training. To further alleviate the mask ambiguity, we present an efficient one-shot fine-tuning variant, PerSAM-F. Freezing the entire SAM, we introduce two learnable weights for multi-scale masks, only training 2 parameters within 10 seconds for improved performance. To demonstrate our efficacy, we construct a new segmentation dataset, PerSeg, for personalized evaluation, and test our methods on video object segmentation with competitive performance. Besides, our approach can also enhance DreamBooth to personalize Stable Diffusion for text-to-image generation, which discards the background disturbance for better target appearance learning. Code is released at https://github.com/ZrrSkywalker/Personalize-SAM

翻译：受大规模数据预训练驱动，Segment Anything Model（SAM）已被证明是一个强大且可提示的框架，革新了分割模型。尽管具有通用性，但在无需人工提示的情况下，针对特定视觉概念定制SAM仍未被充分探索，例如自动分割不同图像中的宠物狗。本文提出一种无需训练的SAM个性化方法，称为PerSAM。仅需一张带有参考掩码的图像，PerSAM首先通过位置先验定位目标概念，并通过三种技术在其他图像或视频中对其进行分割：目标引导注意力、目标语义提示以及级联后细化。通过这种方式，我们无需任何训练即可有效适配SAM用于私人使用。为进一步缓解掩码模糊性问题，我们提出一种高效的单样本微调变体PerSAM-F。冻结整个SAM，我们引入两个可学习权重用于多尺度掩码，仅需在10秒内训练2个参数即可提升性能。为证明有效性，我们构建了用于个性化评估的新分割数据集PerSeg，并在视频目标分割上测试了我们的方法，取得了具有竞争力的性能。此外，我们的方法还能增强DreamBooth，使其个性化Stable Diffusion用于文本到图像生成，摒弃背景干扰以获得更好的目标外观学习。代码已开源至https://github.com/ZrrSkywalker/Personalize-SAM。