Continual learning is a challenging problem in which models need to be trained on non-stationary data across sequential tasks for class-incremental learning. While previous methods have focused on using either regularization or rehearsal-based frameworks to alleviate catastrophic forgetting in image classification, they are limited to a single modality and cannot learn compact class-aware cross-modal representations for continual audio-visual learning. To address this gap, we propose a novel class-incremental grouping network (CIGN) that can learn category-wise semantic features to achieve continual audio-visual learning. Our CIGN leverages learnable audio-visual class tokens and audio-visual grouping to continually aggregate class-aware features. Additionally, it utilizes class tokens distillation and continual grouping to prevent forgetting parameters learned from previous tasks, thereby improving the model's ability to capture discriminative audio-visual categories. We conduct extensive experiments on VGGSound-Instruments, VGGSound-100, and VGG-Sound Sources benchmarks. Our experimental results demonstrate that the CIGN achieves state-of-the-art audio-visual class-incremental learning performance. Code is available at https://github.com/stoneMo/CIGN.

翻译：持续学习是一个具有挑战性的问题，要求模型在跨序列任务的非平稳数据上进行训练，以实现类别增量学习。以往方法主要依赖正则化或重放框架来缓解图像分类中的灾难性遗忘，但仅限于单一模态，无法学习紧凑的类别感知跨模态表示以支持持续视听学习。为解决这一局限，我们提出了一种新颖的类别增量分组网络（CIGN），能够学习类别级语义特征以实现持续视听学习。我们的CIGN利用可学习的视听类标记和视听分组技术持续聚合类别感知特征。此外，它通过类标记蒸馏和持续分组机制来防止遗忘先前任务中习得的参数，从而提升模型捕获判别性视听类别的能力。我们在VGGSound-Instruments、VGGSound-100和VGG-Sound Sources基准上进行了广泛实验。实验结果表明，CIGN在视听类别增量学习性能上达到了最先进水平。代码发布于https://github.com/stoneMo/CIGN。