Domain adaptive semantic segmentation enables robust pixel-wise understanding in real-world driving scenes. Source-free domain adaptation, as a more practical technique, addresses the concerns of data privacy and storage limitations in typical unsupervised domain adaptation methods. It utilizes a well-trained source model and unlabeled target data to achieve adaptation in the target domain. However, in the absence of source data and target labels, current solutions cannot sufficiently reduce the impact of domain shift and fully leverage the information from the target data. In this paper, we propose an end-to-end source-free domain adaptation semantic segmentation method via Importance-Aware and Prototype-Contrast (IAPC) learning. The proposed IAPC framework effectively extracts domain-invariant knowledge from the well-trained source model and learns domain-specific knowledge from the unlabeled target domain. Specifically, considering the problem of domain shift in the prediction of the target domain by the source model, we put forward an importance-aware mechanism for the biased target prediction probability distribution to extract domain-invariant knowledge from the source model. We further introduce a prototype-contrast strategy, which includes a prototype-symmetric cross-entropy loss and a prototype-enhanced cross-entropy loss, to learn target intra-domain knowledge without relying on labels. A comprehensive variety of experiments on two domain adaptive semantic segmentation benchmarks demonstrates that the proposed end-to-end IAPC solution outperforms existing state-of-the-art methods. Code will be made publicly available at https://github.com/yihong-97/Source-free_IAPC.
翻译:域自适应语义分割能够在真实驾驶场景中实现鲁棒的逐像素理解。作为一种更实用的技术,无源域自适应解决了典型无监督域自适应方法中数据隐私和存储限制的问题。它利用训练好的源模型和无标签的目标数据,在目标域中实现自适应。然而,在缺乏源数据和目标标签的情况下,现有方法无法充分减少域偏移的影响,也无法充分利用目标数据的信息。本文提出了一种基于重要性感知与原型对比学习(IAPC)的端到端无源域自适应语义分割方法。所提出的IAPC框架有效提取了训练好的源模型中的域不变知识,并从无标签的目标域中学习域特定知识。具体而言,针对源模型在目标域预测中存在的域偏移问题,我们提出了一种针对有偏目标预测概率分布的重要性感知机制,以从源模型中提取域不变知识。我们进一步引入了一种原型对比策略,包括原型对称交叉熵损失和原型增强交叉熵损失,用于在不依赖标签的情况下学习目标域内知识。在两个域自适应语义分割基准上的全面实验表明,所提出的端到端IAPC方法优于现有最先进方法。代码将公开于https://github.com/yihong-97/Source-free_IAPC。