In recent years, with the rapid advancement of transformer models, transformer-based multimodal architectures have found wide application in various downstream tasks, including but not limited to Image Captioning, Visual Question Answering (VQA), and Image-Text Generation. However, contemporary approaches to Remote Sensing (RS) VQA often involve resource-intensive techniques, such as full fine-tuning of large models or the extraction of image-text features from pre-trained multimodal models, followed by modality fusion using decoders. These approaches demand significant computational resources and time, and a considerable number of trainable parameters are introduced. To address these challenges, we introduce a novel method known as RSAdapter, which prioritizes runtime and parameter efficiency. RSAdapter comprises two key components: the Parallel Adapter and an additional linear transformation layer inserted after each fully connected (FC) layer within the Adapter. This approach not only improves adaptation to pre-trained multimodal models but also allows the parameters of the linear transformation layer to be integrated into the preceding FC layers during inference, reducing inference costs. To demonstrate the effectiveness of RSAdapter, we conduct an extensive series of experiments using three distinct RS-VQA datasets and achieve state-of-the-art results on all three datasets. The code for RSAdapter will be available online at https://github.com/Y-D-Wang/RSAdapter.

翻译：近年来，随着Transformer模型的快速发展，基于Transformer的多模态架构已广泛应用于各类下游任务，包括图像描述、视觉问答（VQA）及图像文本生成等。然而，当前遥感（RS）VQA方法通常采用资源密集型技术，例如大模型的完整微调，或从预训练多模态模型中提取图像文本特征后通过解码器进行模态融合。这些方法需要消耗大量计算资源与时间，并引入大量可训练参数。为应对这些挑战，我们提出一种新颖方法——RSAdapter，该方法以提升运行时效率和参数效率为核心目标。RSAdapter包含两个关键组件：并行适配器（Parallel Adapter）以及插入在适配器中每个全连接（FC）层后的额外线性变换层。该方案不仅能增强对预训练多模态模型的适配能力，还可在推理阶段将线性变换层的参数合并至前序FC层，从而降低推理开销。为验证RSAdapter的有效性，我们基于三种不同的RS-VQA数据集开展了一系列广泛实验，并在所有数据集上取得了最先进的结果。RSAdapter的代码将开源在https://github.com/Y-D-Wang/RSAdapter。