Continual Learning methods are designed to learn new tasks without erasing previous knowledge. However, Continual Learning often requires massive computational power and storage capacity for satisfactory performance. In this paper, we propose a resource-efficient continual learning method called the Elastic Expansion Network (E2Net). Leveraging core subnet distillation and precise replay sample selection, E2Net achieves superior average accuracy and diminished forgetting within the same computational and storage constraints, all while minimizing processing time. In E2Net, we propose Representative Network Distillation to identify the representative core subnet by assessing parameter quantity and output similarity with the working network, distilling analogous subnets within the working network to mitigate reliance on rehearsal buffers and facilitating knowledge transfer across previous tasks. To enhance storage resource utilization, we then propose Subnet Constraint Experience Replay to optimize rehearsal efficiency through a sample storage strategy based on the structures of representative networks. Extensive experiments conducted predominantly on cloud environments with diverse datasets and also spanning the edge environment demonstrate that E2Net consistently outperforms state-of-the-art methods. In addition, our method outperforms competitors in terms of both storage and computational requirements.

翻译：持续学习方法旨在学习新任务的同时不遗忘旧知识。然而，持续学习通常需要庞大的计算能力和存储容量才能获得令人满意的性能。本文提出了一种名为弹性扩展网络（E2Net）的资源高效持续学习方法。通过利用核心子网蒸馏和精准回放样本选择，E2Net在相同的计算和存储约束下实现了更高的平均准确率和更低的遗忘程度，同时最小化处理时间。在E2Net中，我们提出代表性网络蒸馏方法，通过评估参数数量及与工作网络的输出相似性来识别代表性核心子网，在工作网络内蒸馏相似子网以减少对记忆回放缓冲区的依赖，并促进跨先前任务的知识迁移。为提升存储资源利用率，我们进一步提出子网约束经验回放方法，通过基于代表性网络结构的样本存储策略优化回放效率。在云环境（涉及多种数据集）及边缘环境上的大量实验表明，E2Net始终优于现有最优方法。此外，我们的方法在存储和计算需求方面均优于竞争对手。