We focus on the continual learning problem where the tasks arrive sequentially and the aim is to perform well on the newly arrived task without performance degradation on the previously seen tasks. In contrast to the continual learning literature focusing on the centralized setting, we investigate the distributed estimation framework. We consider the well-established distributed learning algorithm COCOA. We derive closed form expressions for the iterations for the overparametrized case. We illustrate the convergence and the error performance of the algorithm based on the over/under-parameterization of the problem. Our results show that depending on the problem dimensions and data generation assumptions, COCOA can perform continual learning over a sequence of tasks, i.e., it can learn a new task without forgetting previously learned tasks, with access only to one task at a time.

翻译：我们聚焦于持续学习问题，其中任务按顺序到达，目标是在新任务上表现良好的同时，不降低先前任务上的性能。与关注集中式设置的持续学习文献不同，我们研究了分布式估计框架。我们考虑了成熟的分布式学习算法COCOA。针对过参数化情况，我们推导出了迭代的闭式表达式。我们根据问题的过参数化/欠参数化程度，展示了算法的收敛性和误差性能。我们的结果表明，根据问题维度和数据生成假设，COCOA可以在一系列任务上执行持续学习，即它能够学习新任务而不遗忘先前学习过的任务，且每次仅访问一个任务。