Open-vocabulary change detection (OVCD) seeks to recognize arbitrary changes of interest by enabling generalization beyond a fixed set of predefined classes. We reformulate OVCD as a two-stage pipeline: first generate class-agnostic change proposals using visual foundation models (VFMs) such as SAM and DINOv2, and then perform category identification with vision-language models (VLMs) such as CLIP. We reveal that category identification errors are the primary bottleneck of OVCD, mainly due to the limited ability of VLMs based on image-text matching to represent fine-grained land-cover categories. To address this, we propose OpenDPR, a training-free vision-centric diffusion-guided prototype retrieval framework. OpenDPR leverages diffusion models to construct diverse prototypes for target categories offline, and to perform similarity retrieval with change proposals in the visual space during inference. The secondary bottleneck lies in change localization, due to the inherent lack of change priors in VFMs. To bridge this gap, we design a spatial-to-change weakly supervised change detection module named S2C to adapt their strong spatial modeling capabilities for change localization. Integrating the pretrained S2C into OpenDPR leads to an optional weakly supervised variant named OpenDPR-W, which further improves OVCD with minimal supervision. Experimental results on four benchmark datasets demonstrate that the proposed methods achieve state-of-the-art performance under both supervision modes. Code is available at https://github.com/guoqi2002/OpenDPR.
翻译:开放词汇变化检测(OVCD)旨在通过实现对预定义固定类别集合之外任意变化目标的泛化识别。我们将OVCD重新构建为两阶段流水线:首先利用视觉基础模型(如SAM和DINOv2)生成类别无关的变化提案,随后采用视觉语言模型(如CLIP)进行类别识别。研究发现,类别识别错误是OVCD的主要瓶颈,这主要源于基于图像-文本匹配的视觉语言模型在表征细粒度地物类别方面的能力局限。为解决该问题,我们提出OpenDPR——一种免训练的视觉中心扩散引导原型检索框架。OpenDPR利用扩散模型离线构建目标类别的多样化原型,并在推理阶段对视觉空间中的变化提案进行相似性检索。次要瓶颈在于变化定位,这是由于视觉基础模型天然缺乏变化先验。为弥补这一不足,我们设计了一种名为S2C的空间到变化弱监督变化检测模块,将其强大的空间建模能力适配至变化定位任务。将预训练的S2C集成至OpenDPR后,可得到可选的弱监督变体OpenDPR-W,该变体在最小监督条件下进一步提升了OVCD性能。在四个基准数据集上的实验结果表明,所提方法在两种监督模式下均达到了最优性能。代码已开源:https://github.com/guoqi2002/OpenDPR。