Domain generalization for semantic segmentation is highly demanded in real applications, where a trained model is expected to work well in previously unseen domains. One challenge lies in the lack of data which could cover the diverse distributions of the possible unseen domains for training. In this paper, we propose a WEb-image assisted Domain GEneralization (WEDGE) scheme, which is the first to exploit the diversity of web-crawled images for generalizable semantic segmentation. To explore and exploit the real-world data distributions, we collect web-crawled images which present large diversity in terms of weather conditions, sites, lighting, camera styles, etc. We also present a method which injects styles of the web-crawled images into training images on-the-fly during training, which enables the network to experience images of diverse styles with reliable labels for effective training. Moreover, we use the web-crawled images with their predicted pseudo labels for training to further enhance the capability of the network. Extensive experiments demonstrate that our method clearly outperforms existing domain generalization techniques.
翻译:语义分割的域泛化在实际应用中具有高度需求,期望训练后的模型能有效处理未见过的目标域。其中一个挑战在于缺乏能够覆盖潜在未见域多样化分布的训练数据。本文提出了一种基于网络图像辅助的域泛化方案(WEDGE),这是首个利用网络爬取图像多样性实现泛化语义分割的工作。为探索并利用真实世界的数据分布,我们收集了在天气条件、地点、光照、相机风格等方面呈现高度多样性的网络爬取图像。同时提出一种方法,在训练过程中动态将网络图像的风格注入训练图像,使网络能够体验具有可靠标签的多样风格图像以实现有效训练。此外,我们还利用网络爬取图像及其预测的伪标签进行训练,进一步增强网络能力。大量实验表明,我们的方法明显优于现有的域泛化技术。