Deep Neural Networks (DNNs) have been ubiquitously adopted in internet of things and are becoming an integral of our daily life. When tackling the evolving learning tasks in real world, such as classifying different types of objects, DNNs face the challenge to continually retrain themselves according to the tasks on different edge devices. Federated continual learning is a promising technique that offers partial solutions but yet to overcome the following difficulties: the significant accuracy loss due to the limited on-device processing, the negative knowledge transfer caused by the limited communication of non-IID data, and the limited scalability on the tasks and edge devices. In this paper, we propose FedKNOW, an accurate and scalable federated continual learning framework, via a novel concept of signature task knowledge. FedKNOW is a client side solution that continuously extracts and integrates the knowledge of signature tasks which are highly influenced by the current task. Each client of FedKNOW is composed of a knowledge extractor, a gradient restorer and, most importantly, a gradient integrator. Upon training for a new task, the gradient integrator ensures the prevention of catastrophic forgetting and mitigation of negative knowledge transfer by effectively combining signature tasks identified from the past local tasks and other clients' current tasks through the global model. We implement FedKNOW in PyTorch and extensively evaluate it against state-of-the-art techniques using popular federated continual learning benchmarks. Extensive evaluation results on heterogeneous edge devices show that FedKNOW improves model accuracy by 63.24% without increasing model training time, reduces communication cost by 34.28%, and achieves more improvements under difficult scenarios such as large numbers of tasks or clients, and training different complex networks.

翻译：深心内网(DNNS)在互联网上被广泛采用,成为我们日常生活的一个组成部分。在应对现实世界中不断演变的学习任务时,例如对不同类型的物体进行分类时,DNNNS面临挑战,根据不同边缘设备的任务不断进行再培训。FDKNOW是一个充满希望的技术,它提供部分解决方案,但又无法克服以下困难:由于有限在互联网上处理有限、由于非IID数据的深度通信有限,以及由于内部偏差导致知识转移的负面影响,以及任务和边缘设备的可扩展性有限。在本文中,我们提出FDKNOW,一个准确和可扩展的精确持续学习框架,通过签字任务的新概念,不断根据不同的解决方案,不断提取和整合对当前任务影响较大的签名任务的知识。FDKNOW的每个客户都由知识提取器、梯度恢复器,以及最重要的是,升级的通信加固化器。在进行新任务培训期间,使用不断的模型培训时,将硬化器化器化器化器化器化器化了当前的任务,从而防止错误化。