Replay-based methods have proved their effectiveness on online continual learning by rehearsing past samples from an auxiliary memory. With many efforts made on improving training schemes based on the memory, however, the information carried by each sample in the memory remains under-investigated. Under circumstances with restricted storage space, the informativeness of the memory becomes critical for effective replay. Although some works design specific strategies to select representative samples, by only employing a small number of original images, the storage space is still not well utilized. To this end, we propose to Summarize the knowledge from the Stream Data (SSD) into more informative samples by distilling the training characteristics of real images. Through maintaining the consistency of training gradients and relationship to the past tasks, the summarized samples are more representative for the stream data compared to the original images. Extensive experiments are conducted on multiple online continual learning benchmarks to support that the proposed SSD method significantly enhances the replay effects. We demonstrate that with limited extra computational overhead, SSD provides more than 3% accuracy boost for sequential CIFAR-100 under extremely restricted memory buffer. Code in https://github.com/vimar-gu/SSD.

翻译：基于回放的方法通过从辅助记忆中重放过去样本，已被证明在在线持续学习中具有有效性。尽管已有大量研究致力于改进基于记忆的训练方案，但记忆中每个样本所携带的信息仍未被充分挖掘。在存储空间受限的情况下，记忆的信息量对于有效回放变得至关重要。尽管部分工作设计了特定的策略来选择代表性样本，但仅使用少量原始图像仍未能充分利用存储空间。为此，我们提出通过蒸馏真实图像的训练特征，将流数据中的知识摘要（SSD）为更具信息量的样本。通过保持训练梯度的一致性以及与过去任务的关联性，这些摘要样本相比原始图像更能代表流数据。我们在多个在线持续学习基准上进行了大量实验，证明所提出的SSD方法能显著增强回放效果。我们证明，在极低内存缓冲区的限制下，SSD仅需少量额外计算开销即可为顺序CIFAR-100任务带来超过3%的精度提升。代码地址：https://github.com/vimar-gu/SSD。