Semantic segmentation techniques have shown significant progress in recent years, but their robustness to real-world perturbations and data samples not seen during training remains a challenge, particularly in safety-critical applications. In this paper, we propose a novel approach to improve the robustness of semantic segmentation techniques by leveraging the synergy between label-to-image generators and image-to-label segmentation models. Specifically, we design and train Robusta, a novel robust conditional generative adversarial network to generate realistic and plausible perturbed or outlier images that can be used to train reliable segmentation models. We conduct in-depth studies of the proposed generative model, assess the performance and robustness of the downstream segmentation network, and demonstrate that our approach can significantly enhance the robustness of semantic segmentation techniques in the face of real-world perturbations, distribution shifts, and out-of-distribution samples. Our results suggest that this approach could be valuable in safety-critical applications, where the reliability of semantic segmentation techniques is of utmost importance and comes with a limited computational budget in inference. We will release our code shortly.

翻译：语义分割技术近年来取得了显著进展，但其对真实世界扰动及训练中未见数据样本的鲁棒性仍是挑战，尤其是在安全关键应用中。本文提出了一种新方法，通过利用标签到图像生成器与图像到标签分割模型之间的协同作用，提升语义分割技术的鲁棒性。具体而言，我们设计并训练了Robusta——一种新颖的鲁棒条件生成对抗网络，用于生成逼真且合理的扰动或异常图像，从而可用于训练可靠的分割模型。我们对所提出的生成模型进行了深入研究，评估了下游分割网络的性能与鲁棒性，并证明我们的方法能显著增强语义分割技术面对真实世界扰动、分布偏移及分布外样本时的鲁棒性。研究结果表明，该方法在语义分割技术可靠性至关重要且推理计算预算有限的安全关键应用中具有重要价值。我们即将公开代码。