Exemplar-based class-incremental learning (CIL) finetunes the model with all samples of new classes but few-shot exemplars of old classes in each incremental phase, where the "few-shot" abides by the limited memory budget. In this paper, we break this "few-shot" limit based on a simple yet surprisingly effective idea: compressing exemplars by downsampling non-discriminative pixels and saving "many-shot" compressed exemplars in the memory. Without needing any manual annotation, we achieve this compression by generating 0-1 masks on discriminative pixels from class activation maps (CAM). We propose an adaptive mask generation model called class-incremental masking (CIM) to explicitly resolve two difficulties of using CAM: 1) transforming the heatmaps of CAM to 0-1 masks with an arbitrary threshold leads to a trade-off between the coverage on discriminative pixels and the quantity of exemplars, as the total memory is fixed; and 2) optimal thresholds vary for different object classes, which is particularly obvious in the dynamic environment of CIL. We optimize the CIM model alternatively with the conventional CIL model through a bilevel optimization problem. We conduct extensive experiments on high-resolution CIL benchmarks including Food-101, ImageNet-100, and ImageNet-1000, and show that using the compressed exemplars by CIM can achieve a new state-of-the-art CIL accuracy, e.g., 4.8 percentage points higher than FOSTER on 10-Phase ImageNet-1000. Our code is available at https://github.com/xfflzl/CIM-CIL.

翻译：基于样本的类增量学习在每个增量阶段使用新类别的全部样本以及旧类别的少量样本微调模型，其中“少量”受限于有限的内存预算。本文通过一个简单却惊人有效的思路打破这一限制：通过下采样非判别性像素压缩样本，并将压缩后的“多量”样本存入内存。无需手动标注，我们通过类激活图生成判别性像素上的0-1掩码实现压缩。我们提出一种自适应掩码生成模型——类增量掩码，明确解决使用类激活图的两大难点：1) 在固定内存总量下，用任意阈值将类激活图的热力图转换为0-1掩码会在判别性像素覆盖率和样本数量之间产生权衡；2) 不同物体类别的最优阈值不同，这在类增量学习的动态环境中尤为明显。我们通过双层优化问题交替优化类增量掩码模型与传统类增量学习模型。在包含Food-101、ImageNet-100和ImageNet-1000的高分辨率类增量学习基准上进行的大量实验表明，使用类增量掩码压缩后的样本可使类增量学习精度达到新高度，例如在10阶段ImageNet-1000上比FOSTER高4.8个百分点。我们的代码开源在https://github.com/xfflzl/CIM-CIL。