In this work, we combine the two paradigms: Federated Learning (FL) and Continual Learning (CL) for text classification task in cloud-edge continuum. The objective of Federated Continual Learning (FCL) is to improve deep learning models over life time at each client by (relevant and efficient) knowledge transfer without sharing data. Here, we address challenges in minimizing inter-client interference while knowledge sharing due to heterogeneous tasks across clients in FCL setup. In doing so, we propose a novel framework, Federated Selective Inter-client Transfer (FedSeIT) which selectively combines model parameters of foreign clients. To further maximize knowledge transfer, we assess domain overlap and select informative tasks from the sequence of historical tasks at each foreign client while preserving privacy. Evaluating against the baselines, we show improved performance, a gain of (average) 12.4\% in text classification over a sequence of tasks using five datasets from diverse domains. To the best of our knowledge, this is the first work that applies FCL to NLP.

翻译：本研究将联邦学习与持续学习两种范式相结合，应用于云边协同环境下的文本分类任务。联邦持续学习的目标是通过（相关且高效的）知识迁移，在不共享数据的前提下，持续提升各客户端深度学习模型在其生命周期内的性能。针对联邦持续学习设置中因客户端任务异质性导致的知识共享时存在客户端间干扰问题，本文提出了一种新型框架——联邦选择性客户端间迁移（FedSeIT），该框架可选择性地组合外部客户端的模型参数。为最大化知识迁移效果，我们在保护隐私的前提下评估领域重叠度，并从各外部客户端的历史任务序列中筛选具有信息量的任务。与基线方法相比，本方法在五个不同领域数据集的任务序列上平均获得12.4%的文本分类性能提升。据我们所知，这是首个将联邦持续学习应用于自然语言处理领域的研究工作。