We analyze continual learning on a sequence of separable linear classification tasks with binary labels. We show theoretically that learning with weak regularization reduces to solving a sequential max-margin problem, corresponding to a special case of the Projection Onto Convex Sets (POCS) framework. We then develop upper bounds on the forgetting and other quantities of interest under various settings with recurring tasks, including cyclic and random orderings of tasks. We discuss several practical implications to popular training practices like regularization scheduling and weighting. We point out several theoretical differences between our continual classification setting and a recently studied continual regression setting.

翻译：我们分析了在具有二元标签的可分离线性分类任务序列上的持续学习。我们从理论上证明，弱正则化下的学习可归结为求解序列最大间隔问题，这对应于投影到凸集（POCS）框架的一个特例。随后，我们针对任务循环出现和随机排序等重复任务场景，建立了遗忘及其他相关指标的上界。我们讨论了若干实用训练策略（如正则化调度与加权）的实践启示，并指出我们提出的持续分类设置与近期研究的持续回归设置在理论上的若干差异。