Continual learning is a sub-field of machine learning, which aims to allow machine learning models to continuously learn on new data, by accumulating knowledge without forgetting what was learned in the past. In this work, we take a step back, and ask: "Why should one care about continual learning in the first place?". We set the stage by surveying recent continual learning papers published at three major machine learning conferences, and show that memory-constrained settings dominate the field. Then, we discuss five open problems in machine learning, and even though they seem unrelated to continual learning at first sight, we show that continual learning will inevitably be part of their solution. These problems are model-editing, personalization, on-device learning, faster (re-)training and reinforcement learning. Finally, by comparing the desiderata from these unsolved problems and the current assumptions in continual learning, we highlight and discuss four future directions for continual learning research. We hope that this work offers an interesting perspective on the future of continual learning, while displaying its potential value and the paths we have to pursue in order to make it successful. This work is the result of the many discussions the authors had at the Dagstuhl seminar on Deep Continual Learning, in March 2023.

翻译：持续学习是机器学习的一个子领域，旨在通过积累知识而不遗忘过去所学内容，使机器学习模型能够持续学习新数据。在本研究中，我们退一步思考：“首先，为什么应该关注持续学习？”我们通过调查近年来在三大机器学习会议上发表的持续学习论文来奠定基础，并表明内存受限设置主导了该领域。随后，我们讨论了机器学习中的五个未解决问题，尽管它们初看似乎与持续学习无关，但我们表明持续学习将不可避免地成为其解决方案的一部分。这些问题包括模型编辑、个性化、设备端学习、更快速的（再）训练以及强化学习。最后，通过比较这些未解决问题的期望特性与当前持续学习中的假设，我们强调并讨论了持续学习研究的四个未来方向。我们希望本研究能为持续学习的未来提供有趣视角，同时展示其潜在价值及为使其成功所须追寻的路径。本研究是作者们于2023年3月在达格施图尔深度持续学习研讨会上多次讨论的成果。