In this paper, we present the Semantic Boundary Conditioned Backbone (SBCB) framework, a simple yet effective training framework that is model-agnostic and boosts segmentation performance, especially around the boundaries. Motivated by the recent development in improving semantic segmentation by incorporating boundaries as auxiliary tasks, we propose a multi-task framework that uses semantic boundary detection (SBD) as an auxiliary task. The SBCB framework utilizes the nature of the SBD task, which is complementary to semantic segmentation, to improve the backbone of the segmentation head. We apply an SBD head that exploits the multi-scale features from the backbone, where the model learns low-level features in the earlier stages, and high-level semantic understanding in the later stages. This head perfectly complements the common semantic segmentation architectures where the features from the later stages are used for classification. We can improve semantic segmentation models without additional parameters during inference by only conditioning the backbone. Through extensive evaluations, we show the effectiveness of the SBCB framework by improving various popular segmentation heads and backbones by 0.5% ~ 3.0% IoU on the Cityscapes dataset and gains 1.6% ~ 4.1% in boundary Fscores. We also apply this framework on customized backbones and the emerging vision transformer models and show the effectiveness of the SBCB framework.

翻译：本文提出了语义边界条件化骨干网络（SBCB）框架，这是一种简单而有效的训练框架，具有模型无关性，能提升分割性能，尤其在边界区域。受近年来通过将边界信息作为辅助任务改进语义分割研究的启发，我们提出了一种多任务框架，将语义边界检测（SBD）作为辅助任务。SBCB框架利用SBD任务与语义分割互补的特性，优化分割头（segmentation head）的骨干网络。我们设计了一个SBD头，通过挖掘骨干网络的多尺度特征，使模型在早期阶段学习低级特征，在后期阶段学习高级语义理解。该头与常见的语义分割架构（通常利用后期阶段特征进行分类）形成完美互补。仅通过条件化骨干网络，我们即可在不增加推理参数的情况下改进语义分割模型。通过大量实验，我们验证了SBCB框架的有效性：在Cityscapes数据集上，该框架使多种主流分割头和骨干网络的IoU提升0.5%~3.0%，边界F值提升1.6%~4.1%。我们还将该框架应用于定制化骨干网络和新兴的视觉Transformer模型，进一步验证了SBCB框架的有效性。