Adversarial robustness has been studied extensively in image classification, especially for the $\ell_\infty$-threat model, but significantly less so for related tasks such as object detection and semantic segmentation, where attacks turn out to be a much harder optimization problem than for image classification. We propose several problem-specific novel attacks minimizing different metrics in accuracy and mIoU. The ensemble of our attacks, SEA, shows that existing attacks severely overestimate the robustness of semantic segmentation models. Surprisingly, existing attempts of adversarial training for semantic segmentation models turn out to be weak or even completely non-robust. We investigate why previous adaptations of adversarial training to semantic segmentation failed and show how recently proposed robust ImageNet backbones can be used to obtain adversarially robust semantic segmentation models with up to six times less training time for PASCAL-VOC and the more challenging ADE20k. The associated code and robust models are available at https://github.com/nmndeep/robust-segmentation

翻译：对抗鲁棒性在图像分类领域已得到广泛研究，尤其是在$\ell_\infty$威胁模型下，但对于目标检测和语义分割等相关任务的研究则显著不足，这些任务的攻击问题被证明是比图像分类困难得多的优化问题。我们提出了几种针对特定问题的新型攻击方法，旨在最小化准确率和mIoU等不同度量指标。我们攻击方法的集成——SEA表明，现有攻击严重高估了语义分割模型的鲁棒性。令人惊讶的是，现有针对语义分割模型的对抗训练尝试被证明是脆弱的，甚至完全不具鲁棒性。我们探究了先前将对抗训练应用于语义分割失败的原因，并展示了如何利用近期提出的鲁棒ImageNet骨干网络来获得对抗鲁棒的语义分割模型，在PASCAL-VOC和更具挑战性的ADE20k数据集上，训练时间最多可减少六倍。相关代码和鲁棒模型可在https://github.com/nmndeep/robust-segmentation获取。