Continual semantic segmentation aims to learn new classes while maintaining the information from the previous classes. Although prior studies have shown impressive progress in recent years, the fairness concern in the continual semantic segmentation needs to be better addressed. Meanwhile, fairness is one of the most vital factors in deploying the deep learning model, especially in human-related or safety applications. In this paper, we present a novel Fairness Continual Learning approach to the semantic segmentation problem. In particular, under the fairness objective, a new fairness continual learning framework is proposed based on class distributions. Then, a novel Prototypical Contrastive Clustering loss is proposed to address the significant challenges in continual learning, i.e., catastrophic forgetting and background shift. Our proposed loss has also been proven as a novel, generalized learning paradigm of knowledge distillation commonly used in continual learning. Moreover, the proposed Conditional Structural Consistency loss further regularized the structural constraint of the predicted segmentation. Our proposed approach has achieved State-of-the-Art performance on three standard scene understanding benchmarks, i.e., ADE20K, Cityscapes, and Pascal VOC, and promoted the fairness of the segmentation model.

翻译：持续语义分割旨在学习新类别的同时保留先前类别的信息。尽管近年来的研究取得了显著进展，但持续语义分割中的公平性问题仍需更好地解决。同时，公平性是部署深度学习模型时最关键的要素之一，尤其在涉及人类或安全的应用场景中。本文提出了一种面向语义分割问题的公平持续学习方法。具体而言，在公平性目标下，我们基于类别分布提出了一种新的公平持续学习框架。随后，我们提出了一种新颖的原型对比聚类损失函数（Prototypical Contrastive Clustering loss），以应对持续学习中的重大挑战——灾难性遗忘和背景偏移。该损失函数被证明是一种新型、通用的知识蒸馏学习范式，适用于持续学习。此外，所提出的条件结构一致性损失函数（Conditional Structural Consistency loss）进一步正则化了预测分割结果的结构约束。我们的方法在三个标准场景理解基准数据集（ADE20K、Cityscapes 和 Pascal VOC）上达到了最先进的性能，并提升了分割模型的公平性。