A robust Multimodal Large Language Model (MLLM) for Earth Observation should maintain consistent interpretation and reasoning under realistic input variations. However, current Remote Sensing MLLMs fail to meet this requirement. Trained on carefully curated clean datasets, they learn brittle mappings that do not generalize to noisy conditions in operational Earth Observation. Consequently, their performance degrades when confronted with imperfect inputs in deployment. To quantify this vulnerability, we construct a realistic set of multimodal perturbations, including visual degradations such as cloud and fog cover, together with diverse human-centric textual variations ranging from colloquialisms to vague or omitted instructions. Empirical evaluations show that these perturbations significantly impair the visual-semantic reasoning capabilities of leading RS foundation models. To address this limitation, we introduce RemoteShield, a robust Remote Sensing MLLM trained to maintain consistent outputs across realistic input variations. During training, each clean sample is paired with its image-text perturbed variants to form a semantic equivalence cluster. Rather than directly fitting noisy samples, RemoteShield is optimized through preference learning over clean and perturbed conditions within the same cluster. By comparing model responses to clean and corrupted inputs, the model is encouraged to favor stable responses over perturbation-induced failures. This cross-condition alignment helps the model focus on underlying task semantics despite visual degradations and textual noise. Experiments on three Earth Observation tasks show that RemoteShield consistently delivers stronger robustness and cross-condition consistency than representative baselines under realistic multimodal perturbations.
翻译:用于地球观测的稳健多模态大语言模型(MLLM)应在现实输入变化下保持一致的解读与推理能力。然而,当前的遥感MLLM未能满足这一要求。由于在精心筛选的清洁数据集上训练,它们学习到的脆弱映射无法泛化至实际地球观测中的噪声条件,导致在部署中面对不完美输入时性能下降。为量化这一脆弱性,我们构建了一套现实的多模态扰动集合,包括云层与雾霾等视觉退化,以及从口语化表达到模糊或缺失指令等多样化的人类中心文本变化。实证评估表明,这些扰动显著损害了主流遥感基础模型的视觉语义推理能力。为解决此局限,我们提出RemoteShield——一种在现实输入变化下维持一致输出的稳健遥感MLLM。在训练过程中,每个清洁样本与其图文扰动变体配对,形成语义等价簇。RemoteShield并非直接拟合噪声样本,而是通过同一簇内清洁与扰动条件下的偏好学习进行优化。通过对比模型对清洁输入与损坏输入的反应,模型被鼓励偏好稳定响应而非扰动导致的失效。这种跨条件对齐使模型能够在视觉退化与文本噪声中聚焦于底层任务语义。在三项地球观测任务上的实验表明,在现实多模态扰动下,RemoteShield相较于代表性基线持续展现出更强的鲁棒性与跨条件一致性。