Continual learning is increasingly sought after in real world machine learning applications, as it enables learning in a more human-like manner. Conventional machine learning approaches fail to achieve this, as incrementally updating the model with non-identically distributed data leads to catastrophic forgetting, where existing representations are overwritten. Although traditional continual learning methods have mostly focused on batch learning, which involves learning from large collections of labeled data sequentially, this approach is not well-suited for real-world applications where we would like new data to be integrated directly. This necessitates a paradigm shift towards streaming learning. In this paper, we propose a streaming version of regularized discriminant analysis as a solution to this challenge. We combine our algorithm with a convolutional neural network and demonstrate that it outperforms both batch learning and existing streaming learning algorithms on the ImageNet ILSVRC-2012 dataset.

翻译：持续学习在现实世界机器学习应用中日益受到青睐，因为它能够以更接近人类的方式进行学习。传统机器学习方法无法实现这一点，因为用非同分布数据增量更新模型会导致灾难性遗忘，即现有表征被覆盖。尽管传统持续学习方法大多聚焦于批学习——即按顺序从大量标注数据中学习，但这种方法并不适合需要直接集成新数据的现实应用场景。这要求向流式学习进行范式转变。本文提出了一种流式正则化判别分析算法来应对这一挑战。我们将该算法与卷积神经网络相结合，并证明其在ImageNet ILSVRC-2012数据集上的表现优于批学习和现有流式学习算法。