In overhead image segmentation tasks, including additional spectral bands beyond the traditional RGB channels can improve model performance. However, it is still unclear how incorporating this additional data impacts model robustness to adversarial attacks and natural perturbations. For adversarial robustness, the additional information could improve the model's ability to distinguish malicious inputs, or simply provide new attack avenues and vulnerabilities. For natural perturbations, the additional information could better inform model decisions and weaken perturbation effects or have no significant influence at all. In this work, we seek to characterize the performance and robustness of a multispectral (RGB and near infrared) image segmentation model subjected to adversarial attacks and natural perturbations. While existing adversarial and natural robustness research has focused primarily on digital perturbations, we prioritize on creating realistic perturbations designed with physical world conditions in mind. For adversarial robustness, we focus on data poisoning attacks whereas for natural robustness, we focus on extending ImageNet-C common corruptions for fog and snow that coherently and self-consistently perturbs the input data. Overall, we find both RGB and multispectral models are vulnerable to data poisoning attacks regardless of input or fusion architectures and that while physically realizable natural perturbations still degrade model performance, the impact differs based on fusion architecture and input data.

翻译：在遥感图像分割任务中，将传统RGB通道以外的光谱波段纳入模型可提升性能。然而，引入额外数据对模型抵御对抗攻击与自然扰动的鲁棒性影响仍不明确。就对抗鲁棒性而言，额外信息可能增强模型区分恶意输入的能力，也可能创造新的攻击途径与脆弱性。就自然扰动而言，额外信息能更好辅助模型决策以削弱扰动影响，或完全不产生显著作用。本研究旨在刻画多光谱（RGB与近红外）图像分割模型在对抗攻击与自然扰动下的性能与鲁棒性特征。现有对抗与自然鲁棒性研究主要关注数字扰动，而我们优先构建基于真实物理世界条件的可行扰动。针对对抗鲁棒性，我们聚焦数据投毒攻击；针对自然鲁棒性，则扩展ImageNet-C常见污染中雾与雪的扰动模式，使其对输入数据进行一致且自洽的扰动。总体而言，研究发现无论输入数据或融合架构如何，RGB与多光谱模型均易受数据投毒攻击；物理可实现的自然扰动虽仍会降低模型性能，但其影响程度因融合架构与输入数据而异。