Change detection (CD) is a critical task to observe and analyze dynamic processes of land cover. Although numerous deep learning-based CD models have performed excellently, their further performance improvements are constrained by the limited knowledge extracted from the given labelled data. On the other hand, the foundation models that emerged recently contain a huge amount of knowledge by scaling up across data modalities and proxy tasks. In this paper, we propose a Bi-Temporal Adapter Network (BAN), which is a universal foundation model-based CD adaptation framework aiming to extract the knowledge of foundation models for CD. The proposed BAN contains three parts, i.e. frozen foundation model (e.g., CLIP), bi-temporal adapter branch (Bi-TAB), and bridging modules between them. Specifically, BAN extracts general features through a frozen foundation model, which are then selected, aligned, and injected into Bi-TAB via the bridging modules. Bi-TAB is designed as a model-agnostic concept to extract task/domain-specific features, which can be either an existing arbitrary CD model or some hand-crafted stacked blocks. Beyond current customized models, BAN is the first extensive attempt to adapt the foundation model to the CD task. Experimental results show the effectiveness of our BAN in improving the performance of existing CD methods (e.g., up to 4.08\% IoU improvement) with only a few additional learnable parameters. More importantly, these successful practices show us the potential of foundation models for remote sensing CD. The code is available at \url{https://github.com/likyoo/BAN} and will be supported in our Open-CD.
翻译:变化检测(CD)是观测与分析地表覆盖动态过程的关键任务。尽管众多基于深度学习的变化检测模型已展现出优异性能,但其性能提升仍受限于从给定标注数据中提取的有限知识。另一方面,近期涌现的基础模型通过跨数据模态与代理任务的大规模扩展,蕴含了海量知识。本文提出一种双时序适配器网络(BAN),这是一种基于基础模型的通用变化检测适应框架,旨在提取基础模型中的知识用于变化检测。所提出的BAN包含三个部分:冻结的基础模型(如CLIP)、双时序适配器分支(Bi-TAB)以及两者之间的桥接模块。具体而言,BAN通过冻结的基础模型提取通用特征,随后经桥接模块对这些特征进行筛选、对齐并注入Bi-TAB。Bi-TAB被设计为一种模型无关的概念,用于提取任务/领域特定特征,其既可以是任意现有的变化检测模型,也可以是手工构建的堆叠模块。不同于现有的定制化模型,BAN是首个将基础模型适配至变化检测任务的广泛尝试。实验结果表明,BAN仅需少量额外可学习参数即可有效提升现有变化检测方法的性能(例如,交并比最高提升4.08%)。更重要的是,这些成功实践展示了基础模型在遥感变化检测中的潜力。相关代码已在 \url{https://github.com/likyoo/BAN} 公开,并将集成至我们的Open-CD框架中。