The performance of a lifelong learning (L3) model degrades when it is trained on a series of tasks, as the geometrical formation of the embedding space changes while learning novel concepts sequentially. The majority of existing L3 approaches operate on a fixed-curvature (e.g., zero-curvature Euclidean) space that is not necessarily suitable for modeling the complex geometric structure of data. Furthermore, the distillation strategies apply constraints directly on low-dimensional embeddings, discouraging the L3 model from learning new concepts by making the model highly stable. To address the problem, we propose a distillation strategy named L3DMC that operates on mixed-curvature spaces to preserve the already-learned knowledge by modeling and maintaining complex geometrical structures. We propose to embed the projected low dimensional embedding of fixed-curvature spaces (Euclidean and hyperbolic) to higher-dimensional Reproducing Kernel Hilbert Space (RKHS) using a positive-definite kernel function to attain rich representation. Afterward, we optimize the L3 model by minimizing the discrepancies between the new sample representation and the subspace constructed using the old representation in RKHS. L3DMC is capable of adapting new knowledge better without forgetting old knowledge as it combines the representation power of multiple fixed-curvature spaces and is performed on higher-dimensional RKHS. Thorough experiments on three benchmarks demonstrate the effectiveness of our proposed distillation strategy for medical image classification in L3 settings. Our code implementation is publicly available at https://github.com/csiro-robotics/L3DMC.

翻译：当终身学习模型在连续任务上训练时，由于顺序学习新概念过程中嵌入空间的几何结构发生变化，其性能会逐渐下降。现有大多数终身学习方法在固定曲率（如零曲率的欧几里得空间）上运行，这种空间未必适合建模数据的复杂几何结构。此外，现有蒸馏策略直接对低维嵌入施加约束，通过使模型高度稳定而阻碍其学习新概念。为解决这一问题，我们提出了一种名为L3DMC的蒸馏策略，该策略在混合曲率空间中运作，通过建模并保持复杂的几何结构来保留已学知识。我们提出利用正定核函数将固定曲率空间（欧几里得空间和双曲空间）的投影低维嵌入映射到高维再生核希尔伯特空间（RKHS），以获得丰富的表征。随后，通过最小化新样本表征与旧表征在RKHS中构建的子空间之间的差异来优化终身学习模型。L3DMC结合了多个固定曲率空间的表征能力，并在高维RKHS中执行，因此能够更好地适应新知识而不会遗忘旧知识。在三个基准数据集上的充分实验证明了我们提出的蒸馏策略在终身学习场景下用于医学图像分类的有效性。我们的代码实现已在https://github.com/csiro-robotics/L3DMC 公开提供。