Recent advancements in foundation models (FMs), such as GPT-4 and LLaMA, have attracted significant attention due to their exceptional performance in zero-shot learning scenarios. Similarly, in the field of visual learning, models like Grounding DINO and the Segment Anything Model (SAM) have exhibited remarkable progress in open-set detection and instance segmentation tasks. It is undeniable that these FMs will profoundly impact a wide range of real-world visual learning tasks, ushering in a new paradigm shift for developing such models. In this study, we concentrate on the remote sensing domain, where the images are notably dissimilar from those in conventional scenarios. We developed a pipeline that leverages multiple FMs to facilitate remote sensing image semantic segmentation tasks guided by text prompt, which we denote as Text2Seg. The pipeline is benchmarked on several widely-used remote sensing datasets, and we present preliminary results to demonstrate its effectiveness. Through this work, we aim to provide insights into maximizing the applicability of visual FMs in specific contexts with minimal model tuning. The code is available at https://github.com/Douglas2Code/Text2Seg.

翻译：近期，GPT-4和LLaMA等基础模型在零样本学习场景中展现出卓越性能，引起了广泛关注。同样，在视觉学习领域，Grounding DINO和Segment Anything Model（SAM）等模型在开放集检测和实例分割任务中取得了显著进展。毋庸置疑，这些基础模型将对广泛的真实视觉学习任务产生深远影响，为该类模型的开发带来新的范式转变。本研究聚焦于遥感领域，该领域的图像与传统场景图像存在显著差异。我们开发了一种利用多个基础模型实现文本提示引导的遥感图像语义分割任务的流程，并将其命名为Text2Seg。该流程在多个广泛使用的遥感数据集上进行了基准测试，我们展示了初步结果以证明其有效性。通过这项工作，我们旨在揭示如何在特定场景中以最少的模型调优最大化视觉基础模型的适用性。代码已开源：https://github.com/Douglas2Code/Text2Seg。