Continual Learning in semantic scene segmentation aims to continually learn new unseen classes in dynamic environments while maintaining previously learned knowledge. Prior studies focused on modeling the catastrophic forgetting and background shift challenges in continual learning. However, fairness, another major challenge that causes unfair predictions leading to low performance among major and minor classes, still needs to be well addressed. In addition, prior methods have yet to model the unknown classes well, thus resulting in producing non-discriminative features among unknown classes. This paper presents a novel Fairness Learning via Contrastive Attention Approach to continual learning in semantic scene understanding. In particular, we first introduce a new Fairness Contrastive Clustering loss to address the problems of catastrophic forgetting and fairness. Then, we propose an attention-based visual grammar approach to effectively model the background shift problem and unknown classes, producing better feature representations for different unknown classes. Through our experiments, our proposed approach achieves State-of-the-Art (SOTA) performance on different continual learning settings of three standard benchmarks, i.e., ADE20K, Cityscapes, and Pascal VOC. It promotes the fairness of the continual semantic segmentation model.

翻译：摘要：语义场景分割中的持续学习旨在动态环境中持续学习未见新类别，同时保留先前习得的知识。以往研究主要聚焦于建模持续学习中的灾难性遗忘和背景偏移挑战。然而，公平性作为导致主类别与次类别之间预测偏差、从而降低模型性能的另一重大挑战，尚未得到充分解决。此外，现有方法对未知类别的建模仍不完善，导致无法为不同未知类别生成具有判别性的特征。本文提出了一种新颖的基于对比注意力机制的公平性学习方法，用于语义场景理解的持续学习。具体而言，我们首先引入一种新的公平性对比聚类损失函数，以解决灾难性遗忘与公平性问题；随后提出基于注意力的视觉语法方法，有效建模背景偏移问题与未知类别，为不同未知类别生成更优的特征表示。实验结果表明，我们的方法在三个标准基准（ADE20K、Cityscapes及Pascal VOC）的多种持续学习设定下均达到了最先进的性能，并有效提升了持续语义分割模型的公平性。