Pre-trained Vision-Language Models (VLMs) utilizing extensive image-text paired data have demonstrated unprecedented image-text association capabilities, achieving remarkable results across various downstream tasks. A critical challenge is how to make use of existing large-scale pre-trained VLMs, which are trained on common objects, to perform the domain-specific transfer for accomplishing domain-related downstream tasks. A critical challenge is how to make use of existing large-scale pre-trained VLMs, which are trained on common objects, to perform the domain-specific transfer for accomplishing domain-related downstream tasks. In this paper, we propose a new framework that includes the Domain pre-trained Vision-Language Model (DVLM), bridging the gap between the General Vision-Language Model (GVLM) and domain-specific downstream tasks. Moreover, we present an image-text paired dataset in the field of remote sensing (RS), RS5M, which has 5 million RS images with English descriptions. The dataset is obtained from filtering publicly available image-text paired datasets and captioning label-only RS datasets with pre-trained VLM. These constitute the first large-scale RS image-text paired dataset. Additionally, we fine-tuned the CLIP model and tried several Parameter-Efficient Fine-Tuning methods on RS5M to implement the DVLM. Experimental results show that our proposed dataset is highly effective for various tasks, and our model GeoRSCLIP improves upon the baseline or previous state-of-the-art model by $3\%\sim20\%$ in Zero-shot Classification (ZSC), $3\%\sim6\%$ in Remote Sensing Cross-Modal Text-Image Retrieval (RSCTIR) and $4\%\sim5\%$ in Semantic Localization (SeLo) tasks. Dataset and models have been released in: \url{https://github.com/om-ai-lab/RS5M}.

翻译：利用大规模图像-文本配对数据的预训练视觉-语言模型（VLM）展现了前所未有的图像-文本关联能力，在各类下游任务中取得了显著成果。一个关键挑战在于如何将基于通用物体训练的大规模预训练VLM进行领域特定迁移，以完成领域相关的下游任务。本文提出了一种新框架——领域预训练视觉-语言模型（DVLM），用以弥合通用视觉-语言模型（GVLM）与特定领域下游任务之间的鸿沟。此外，我们构建了遥感领域的图像-文本配对数据集RS5M，包含500万张带有英文描述的遥感图像。该数据集通过筛选公开图像-文本配对数据集以及利用预训练VLM对仅含标签的遥感数据集进行描述生成而获得，构成首个大规模遥感图像-文本配对数据集。我们进一步在RS5M上对CLIP模型进行微调，并尝试了多种参数高效微调方法以实现DVLM。实验结果表明，所提数据集对各类任务高度有效；我们的模型GeoRSCLIP在零样本分类（ZSC）任务上相比基线或先前最先进模型提升3%~20%，在遥感跨模态文本-图像检索（RSCTIR）任务上提升3%~6%，在语义定位（SeLo）任务上提升4%~5%。数据集与模型已发布于：\url{https://github.com/om-ai-lab/RS5M}。