Remote Sensing Image Change Captioning (RSICC) aims to describe surface changes between multi-temporal remote sensing images in language, including the changed object categories, locations, and dynamics of changing objects (e.g., added or disappeared). This poses challenges to spatial and temporal modeling of bi-temporal features. Despite previous methods progressing in the spatial change perception, there are still weaknesses in joint spatial-temporal modeling. To address this, in this paper, we propose a novel RSCaMa model, which achieves efficient joint spatial-temporal modeling through multiple CaMa layers, enabling iterative refinement of bi-temporal features. To achieve efficient spatial modeling, we introduce the recently popular Mamba (a state space model) with a global receptive field and linear complexity into the RSICC task and propose the Spatial Difference-aware SSM (SD-SSM), overcoming limitations of previous CNN- and Transformer-based methods in the receptive field and computational complexity. SD-SSM enhances the model's ability to capture spatial changes sharply. In terms of efficient temporal modeling, considering the potential correlation between the temporal scanning characteristics of Mamba and the temporality of the RSICC, we propose the Temporal-Traversing SSM (TT-SSM), which scans bi-temporal features in a temporal cross-wise manner, enhancing the model's temporal understanding and information interaction. Experiments validate the effectiveness of the efficient joint spatial-temporal modeling and demonstrate the outstanding performance of RSCaMa and the potential of the Mamba in the RSICC task. Additionally, we systematically compare three different language decoders, including Mamba, GPT-style decoder, and Transformer decoder, providing valuable insights for future RSICC research. The code will be available at \emph{\url{https://github.com/Chen-Yang-Liu/RSCaMa}}
翻译:遥感影像变化描述(RSICC)旨在用语言描述多时相遥感影像之间的地表变化,包括变化对象的类别、位置及变化对象的动态(例如新增或消失)。这对双时相特征的空域与时域建模提出了挑战。尽管先前方法在空间变化感知方面取得了进展,但在联合时空建模方面仍存在不足。为此,本文提出了一种新颖的RSCaMa模型,通过多个CaMa层实现高效的联合时空建模,从而对双时相特征进行迭代优化。为实现高效的空域建模,我们将近期流行的Mamba(一种具有全局感受野和线性复杂度的状态空间模型)引入RSICC任务,并提出了空间差异感知SSM(SD-SSM),克服了先前基于CNN和Transformer的方法在感受野和计算复杂度上的局限性。SD-SSM显著增强了模型捕捉空间变化的能力。在高效的时域建模方面,考虑到Mamba的时域扫描特性与RSICC时域性之间的潜在关联,我们提出了时域遍历SSM(TT-SSM),它以时域交叉方式扫描双时相特征,增强了模型的时域理解能力与信息交互。实验验证了高效联合时空建模的有效性,并展示了RSCaMa的卓越性能以及Mamba在RSICC任务中的潜力。此外,我们系统比较了三种不同的语言解码器(包括Mamba、GPT风格解码器和Transformer解码器),为未来RSICC研究提供了宝贵见解。代码将发布于\emph{\url{https://github.com/Chen-Yang-Liu/RSCaMa}}。