Online continual learning aims to get closer to a live learning experience by learning directly on a stream of data with temporally shifting distribution and by storing a minimum amount of data from that stream. In this empirical evaluation, we evaluate various methods from the literature that tackle online continual learning. More specifically, we focus on the class-incremental setting in the context of image classification, where the learner must learn new classes incrementally from a stream of data. We compare these methods on the Split-CIFAR100 and Split-TinyImagenet benchmarks, and measure their average accuracy, forgetting, stability, and quality of the representations, to evaluate various aspects of the algorithm at the end but also during the whole training period. We find that most methods suffer from stability and underfitting issues. However, the learned representations are comparable to i.i.d. training under the same computational budget. No clear winner emerges from the results and basic experience replay, when properly tuned and implemented, is a very strong baseline. We release our modular and extensible codebase at https://github.com/AlbinSou/ocl_survey based on the avalanche framework to reproduce our results and encourage future research.

翻译：在线持续学习旨在通过直接处理具有时变分布的数据流并最小化存储数据量，以更贴近实时学习体验。在本实证评估中，我们系统评估了文献中提出的多种在线持续学习方法，重点聚焦图像分类场景下的类增量学习设置——学习器需从数据流中逐步学习新类别。我们在Split-CIFAR100和Split-TinyImagenet基准上比较了这些方法，通过测量平均准确率、遗忘程度、稳定性及表征质量，不仅评估算法在训练终点的表现，更追踪整个训练周期的动态变化。实验发现，多数方法存在稳定性与欠拟合问题，但学习到的表征在相同计算预算下与独立同分布训练相当。各方法间未出现明确优胜者，而基础经验回放方法经合理调参实现后展现出极强的基准性能。我们基于avalanche框架发布了模块化可扩展代码库（https://github.com/AlbinSou/ocl_survey），以支持结果复现并促进后续研究。