With autonomous industries on the rise, domain adaptation of the visual perception stack is an important research direction due to the cost savings promise. Much prior art was dedicated to domain-adaptive semantic segmentation in the synthetic-to-real context. Despite being a crucial output of the perception stack, panoptic segmentation has been largely overlooked by the domain adaptation community. Therefore, we revisit well-performing domain adaptation strategies from other fields, adapt them to panoptic segmentation, and show that they can effectively enhance panoptic domain adaptation. Further, we study the panoptic network design and propose a novel architecture (EDAPS) designed explicitly for domain-adaptive panoptic segmentation. It uses a shared, domain-robust transformer encoder to facilitate the joint adaptation of semantic and instance features, but task-specific decoders tailored for the specific requirements of both domain-adaptive semantic and instance segmentation. As a result, the performance gap seen in challenging panoptic benchmarks is substantially narrowed. EDAPS significantly improves the state-of-the-art performance for panoptic segmentation UDA by a large margin of 20% on SYNTHIA-to-Cityscapes and even 72% on the more challenging SYNTHIA-to-Mapillary Vistas. The implementation is available at https://github.com/susaha/edaps.
翻译:随着自主产业的崛起,视觉感知栈的域适应因具成本节约潜力而成为重要研究方向。先前大量研究聚焦于合成到真实场景下的域自适应语义分割。然而,全景分割作为感知栈的关键输出却被域适应社区广泛忽视。为此,我们重新审视其他领域效果良好的域适应策略,将其适配于全景分割,并证明这些策略能有效增强全景域适应。进一步,我们研究全景网络设计,提出一种专为域自适应全景分割设计的新型架构(EDAPS)。该架构采用共享的、对域鲁棒的Transformer编码器以促进语义与实例特征的联合适应,同时为域自适应语义分割和实例分割的特定需求分别定制任务专用解码器。这使得具有挑战性的全景基准测试中的性能差距显著缩小。EDAPS将全景分割无监督域适应(UDA)的最优性能大幅提升:在SYNTHIA-to-Cityscapes上提升20%,在更具挑战性的SYNTHIA-to-Mapillary Vistas上甚至提升72%。实现代码见https://github.com/susaha/edaps。