Continual Learning (CL) is a process in which there is still huge gap between human and deep learning model efficiency. Recently, many CL algorithms were designed. Most of them have many problems with learning in dynamic and complex environments. In this work new architecture based approach Ada-QPacknet is described. It incorporates the pruning for extracting the sub-network for each task. The crucial aspect in architecture based CL methods is theirs capacity. In presented method the size of the model is reduced by efficient linear and nonlinear quantisation approach. The method reduces the bit-width of the weights format. The presented results shows that low bit quantisation achieves similar accuracy as floating-point sub-network on a well-know CL scenarios. To our knowledge it is the first CL strategy which incorporates both compression techniques pruning and quantisation for generating task sub-networks. The presented algorithm was tested on well-known episode combinations and compared with most popular algorithms. Results show that proposed approach outperforms most of the CL strategies in task and class incremental scenarios.

翻译：持续学习（Continual Learning, CL）是一个人类效率与深度学习模型之间仍存在巨大差距的过程。近年来，许多CL算法被设计出来，但其中大多数在学习动态复杂环境时面临诸多问题。本文描述了一种基于架构的新方法Ada-QPacknet。该方法通过剪枝为每个任务提取子网络。在基于架构的CL方法中，其容量是关键因素。所提方法通过高效的线性与非线性量化方法降低了模型规模，并缩减了权重格式的位宽。实验结果表明，在经典CL场景中，低位量化能达到与浮点子网络相似的准确率。据我们所知，这是首个融合剪枝与量化两种压缩技术来生成任务子网络的CL策略。该算法在著名情节组合上进行了测试，并与主流算法进行了比较。结果显示，所提方法在任务增量与类别增量场景中均优于多数CL策略。