Segment anything model (SAM), as the name suggests, is claimed to be capable of cutting out any object. SAM is a vision foundation model which demonstrates impressive zero-shot transfer performance with the guidance of a prompt. However, there is currently a lack of comprehensive evaluation of its robustness performance under various types of corruptions. Prior works show that SAM is biased towards texture (style) rather than shape, motivated by which we start by investigating SAM's robustness against style transfer, which is synthetic corruption. With the effect of corruptions interpreted as a style change, we further evaluate its robustness on 15 common corruptions with 5 severity levels for each real-world corruption. Beyond the corruptions, we further evaluate the SAM robustness on local occlusion and adversarial perturbations. Overall, this work provides a comprehensive empirical study on the robustness of the SAM under corruptions and beyond.
翻译:分段一切模型(SAM)顾名思义能够分割任意目标。作为一种视觉基础模型,SAM在提示引导下展现出令人瞩目的零样本迁移性能。然而,目前尚缺乏对其在各种类型扰动下鲁棒性的全面评估。已有研究表明SAM更偏向纹理(风格)而非形状特征,受此启发,我们首先探究SAM对风格迁移(一种合成扰动)的鲁棒性。将扰动效应理解为风格变化后,我们进一步评估其在15种常见真实扰动(每种含5个严重级别)下的表现。除扰动外,我们还评估了SAM对局部遮挡和对抗性扰动的鲁棒性。总体而言,本研究对SAM模型在扰动及其扩展场景下的鲁棒性进行了全面的实证分析。