VLMs trained on web-scale data retain sensitive and copyrighted visual concepts that deployment may require removing. Training-based unlearning methods share a structural flaw: fine-tuning on a narrow forget set degrades general capabilities before unlearning begins, making it impossible to attribute subsequent performance drops to the unlearning procedure itself. Training-free approaches sidestep this by suppressing concepts through prompts or system instructions, but no rigorous benchmark exists for evaluating them on visual tasks. We introduce VLM-UnBench, the first benchmark for training-free visual concept unlearning in VLMs. It covers four forgetting levels, 7 source datasets, and 11 concept axes, and pairs a three-level probe taxonomy with five evaluation conditions to separate genuine forgetting from instruction compliance. Across 8 evaluation settings and 13 VLM configurations, realistic unlearning prompts leave forget accuracy near the no-instruction baseline; meaningful reductions appear only under oracle conditions that disclose the target concept to the model. Object and scene concepts are the most resistant to suppression, and stronger instruction-tuned models remain capable despite explicit forget instructions. These results expose a clear gap between prompt-level suppression and true visual concept erasure.

翻译：在Web规模数据上训练的VLMs会保留敏感及受版权保护的视觉概念，而部署时可能需移除这些概念。基于训练过程的遗忘方法存在结构性缺陷：在遗忘开始前，针对窄遗忘集的微调会先导致通用能力退化，使得后续性能下降无法归因于遗忘机制本身。无训练方法通过提示词或系统指令抑制概念来规避此问题，但目前缺乏严格基准来评估其在视觉任务上的表现。我们提出VLM-UnBench，这是首个面向VLMs无训练视觉概念遗忘的基准测试。该基准涵盖四个遗忘层级、7个源数据集和11个概念维度，并采用三级探针分类体系与五类评估条件，用以区分真实遗忘与指令遵从。在8种评估设置与13种VLM配置下，现实遗忘提示仅使遗忘集准确率接近无指令基线水平；仅在向模型揭示目标概念的预言条件下才出现显著下降。物体与场景概念的抑制抗性最强，且经过更强指令微调的模型即便收到显式遗忘指令仍保持能力。这些结果揭示了提示级抑制与真实视觉概念擦除之间的显著差距。