Pre-trained Vision-Language Models (VLMs) utilizing extensive image-text paired data have demonstrated unprecedented image-text association capabilities, achieving remarkable results across various downstream tasks. A critical challenge is how to make use of existing large-scale pre-trained VLMs, which are trained on common objects, to perform the domain-specific transfer for accomplishing domain-related downstream tasks. A critical challenge is how to make use of existing large-scale pre-trained VLMs, which are trained on common objects, to perform the domain-specific transfer for accomplishing domain-related downstream tasks. In this paper, we propose a new framework that includes the Domain pre-trained Vision-Language Model (DVLM), bridging the gap between the General Vision-Language Model (GVLM) and domain-specific downstream tasks. Moreover, we present an image-text paired dataset in the field of remote sensing (RS), RS5M, which has 5 million RS images with English descriptions. The dataset is obtained from filtering publicly available image-text paired datasets and captioning label-only RS datasets with pre-trained VLM. These constitute the first large-scale RS image-text paired dataset. Additionally, we fine-tuned the CLIP model and tried several Parameter-Efficient Fine-Tuning methods on RS5M to implement the DVLM. Experimental results show that our proposed dataset is highly effective for various tasks, and our model GeoRSCLIP improves upon the baseline or previous state-of-the-art model by $3\%\sim20\%$ in Zero-shot Classification (ZSC), $3\%\sim6\%$ in Remote Sensing Cross-Modal Text-Image Retrieval (RSCTIR) and $4\%\sim5\%$ in Semantic Localization (SeLo) tasks. Dataset and models have been released in: \url{https://github.com/om-ai-lab/RS5M}.

翻译：基于大规模图文对数据预训练的视觉-语言模型（VLMs）展现出前所未有的图像-文本关联能力，在各类下游任务中取得了显著成果。一个关键挑战在于如何利用现有基于通用物体预训练的大规模VLMs进行领域特定迁移，以完成领域相关的下游任务。本文提出一种新框架，包含领域预训练视觉-语言模型（DVLM），以弥合通用视觉-语言模型（GVLM）与领域特定下游任务之间的鸿沟。此外，我们构建了遥感（RS）领域的图文对数据集RS5M，包含500万张带英文描述的遥感图像。该数据集通过筛选公开图文对数据集以及利用预训练VLM对仅含标签的遥感数据集进行描述生成而获得，由此构成了首个大规模遥感图文对数据集。我们进一步在RS5M上微调CLIP模型，并尝试多种参数高效微调方法以构建DVLM。实验结果表明，所提数据集在多个任务中效果显著：我们的GeoRSCLIP模型在零样本分类（ZSC）任务上较基线或先前最优模型提升3%~20%，在遥感跨模态文本-图像检索（RSCTIR）任务上提升3%~6%，在语义定位（SeLo）任务上提升4%~5%。数据集及模型已发布于：\url{https://github.com/om-ai-lab/RS5M}。