Domain generalized semantic segmentation (DGSS) is an essential but highly challenging task, in which the model is trained only on source data and any target data is not available. Existing DGSS methods primarily standardize the feature distribution or utilize extra domain data for augmentation. However, the former sacrifices valuable information and the latter introduces domain biases. Therefore, generating diverse-style source data without auxiliary data emerges as an attractive strategy. In light of this, we propose GAN-based feature augmentation (GBFA) that hallucinates stylized feature maps while preserving their semantic contents with a feature generator. The impressive generative capability of GANs enables GBFA to perform inter-channel and trainable feature synthesis in an end-to-end framework. To enable learning GBFA, we introduce random image color augmentation (RICA), which adds a diverse range of variations to source images during training. These augmented images are then passed through a feature extractor to obtain features tailored for GBFA training. Both GBFA and RICA operate exclusively within the source domain, eliminating the need for auxiliary datasets. We conduct extensive experiments, and the generalization results from the synthetic GTAV and SYNTHIA to the real Cityscapes, BDDS, and Mapillary datasets show that our method achieves state-of-the-art performance in DGSS.
翻译:域泛化语义分割(DGSS)是一项重要但极具挑战性的任务,其中模型仅在源域数据上训练,且无法获取任何目标域数据。现有DGSS方法主要通过标准化特征分布或利用额外域数据进行数据增强来提升泛化性能。然而,前者会损失有价值的信息,后者则会引入域偏差。因此,在不依赖辅助数据的情况下生成多样风格的源域数据成为一种有吸引力的策略。基于此,我们提出基于生成对抗网络的特征增强方法(GBFA),该方法通过特征生成器合成了风格化特征图,同时保留其语义内容。GAN的卓越生成能力使GBFA能够在端到端框架中实现跨通道且可训练的特征合成。为训练GBFA,我们引入随机图像色彩增强(RICA),该技术在训练过程中为源图像添加多样化的色彩变化。这些增强图像随后通过特征提取器获得专用于GBFA训练的特征。GBFA与RICA均在源域内运行,无需任何辅助数据集。我们进行了大量实验,从合成数据集GTAV和SYNTHIA到真实数据集Cityscapes、BDDS和Mapillary的泛化结果表明,该方法在DGSS任务中达到了最先进的性能。