Semantic segmentation is an important task for scene understanding in self-driving cars and robotics, which aims to assign dense labels for all pixels in the image. Existing work typically improves semantic segmentation performance by exploring different network architectures on a target dataset. Little attention has been paid to build a unified system by simultaneously learning from multiple datasets due to the inherent distribution shift across different datasets. In this paper, we propose a simple, flexible, and general method for semantic segmentation, termed Cross-Dataset Collaborative Learning (CDCL). Our goal is to train a unified model for improving the performance in each dataset by leveraging information from all the datasets. Specifically, we first introduce a family of Dataset-Aware Blocks (DAB) as the fundamental computing units of the network, which help capture homogeneous convolutional representations and heterogeneous statistics across different datasets. Second, we present a Dataset Alternation Training (DAT) mechanism to facilitate the collaborative optimization procedure. We conduct extensive evaluations on diverse semantic segmentation datasets for autonomous driving. Experiments demonstrate that our method consistently achieves notable improvements over prior single-dataset and cross-dataset training methods without introducing extra FLOPs. Particularly, with the same architecture of PSPNet (ResNet-18), our method outperforms the single-dataset baseline by 5.65\%, 6.57\%, and 5.79\% mIoU on the validation sets of Cityscapes, BDD100K, CamVid, respectively. We also apply CDCL for point cloud 3D semantic segmentation and achieve improved performance, which further validates the superiority and generality of our method. Code and models will be released.

翻译：语义分解是一项重要任务, 用于自我驱动汽车和机器人的现场理解, 目的是为图像中的所有像素指定密度标签。 现有工作通常通过在目标数据集中探索不同的网络结构来改进语义分解性能。 由于不同数据集之间固有的分布变化, 很少注意同时从多个数据集中学习, 以构建一个统一的系统。 在本文中, 我们提出了一个简单、 灵活和通用的语义分解方法, 称为 Cross- Datas 合作性学习( CDCL ) 。 我们的目标是通过利用所有数据集的信息来培训一个统一模型来改进每个数据集的性能。 具体地说, 我们首先将数据集- Award 区块( DAB ) 作为网络的基本计算单位, 帮助在不同数据集中采集同质的演算图和混杂统计数据。 其次, 我们提出一个数据集解解( DAT) 机制, 以便利协作优化程序。 我们对各种语义分解数据数据集数据集进行广泛的评估, 用于自主驱动。 5, 实验我们的方法持续地在先前的Sal- diadeal dal rual rudeal 校正 中实现超前 CD- sal 。