Replay-based methods have proved their effectiveness on online continual learning by rehearsing past samples from an auxiliary memory. With many efforts made on improving training schemes based on the memory, however, the information carried by each sample in the memory remains under-investigated. Under circumstances with restricted storage space, the informativeness of the memory becomes critical for effective replay. Although some works design specific strategies to select representative samples, by only employing original images, the storage space is still not well utilized. To this end, we propose to Summarize the knowledge from the Stream Data (SSD) into more informative samples by distilling the training characteristics of real images. Through maintaining the consistency of training gradients and relationship to the past tasks, the summarized samples are more representative for the stream data compared to the original images. Extensive experiments are conducted on multiple online continual learning benchmarks to support that the proposed SSD method significantly enhances the replay effects. We demonstrate that with limited extra computational overhead, SSD provides more than 3% accuracy boost for sequential CIFAR-100 under extremely restricted memory buffer. The code is available in https://github.com/vimar-gu/SSD.

翻译：基于回放的方法通过辅助记忆体中复现历史样本，在在线连续学习中展现出显著效果。尽管现有研究致力于改进基于记忆体的训练方案，但记忆体中每个样本承载的信息尚未得到充分探索。在存储空间受限的背景下，记忆体的信息密度对于高效回放至关重要。虽然已有工作设计了特定策略来筛选代表性样本，但仅采用原始图像仍未能充分利用存储空间。为此，我们提出通过提炼真实图像的训练特性，将流数据中的知识摘要为信息密度更高的样本。通过保持训练梯度一致性以及与历史任务的关联性，摘要所得样本相比原始图像具有更强的流数据表征能力。在多个在线连续学习基准上的大量实验表明，所提出的SSD方法能显著增强回放效果。实验证明，在极小内存缓冲区限制下，SSD方法仅需有限额外计算开销即可为CIFAR-100顺序学习任务带来超过3%的准确率提升。代码已开源至https://github.com/vimar-gu/SSD。