Continual learning with an increasing number of classes is a challenging task. The difficulty rises when each example is presented exactly once, which requires the model to learn online. Recent methods with classic parameter optimization procedures have been shown to struggle in such setups or have limitations like non-differentiable components or memory buffers. For this reason, we present the fully differentiable ensemble method that allows us to efficiently train an ensemble of neural networks in the end-to-end regime. The proposed technique achieves SOTA results without a memory buffer and clearly outperforms the reference methods. The conducted experiments have also shown a significant increase in the performance for small ensembles, which demonstrates the capability of obtaining relatively high classification accuracy with a reduced number of classifiers.

翻译：持续学习不断增多的类别是一项具有挑战性的任务。当每个样本仅出现一次时，难度进一步增加，这要求模型能够进行在线学习。研究表明，采用经典参数优化方法的最新方法在此类场景中表现不佳，或存在诸如不可微分组件或内存缓冲区等局限性。为此，我们提出了一种全可微分的集成方法，能够在端到端框架下高效训练神经网络集成。所提出的技术无需内存缓冲区即可达到最新水平（SOTA）结果，并明显优于参考方法。进行的实验还表明，小型集成在性能上显著提升，展现了在减少分类器数量的情况下仍能获得较高分类精度的能力。