Catastrophic forgetting, the phenomenon of forgetting previously learned tasks when learning a new one, is a major hurdle in developing continual learning algorithms. A popular method to alleviate forgetting is to use a memory buffer, which stores a subset of previously learned task examples for use during training on new tasks. The de facto method of filling memory is by randomly selecting previous examples. However, this process could introduce outliers or noisy samples that could hurt the generalization of the model. This paper introduces Memory Outlier Elimination (MOE), a method for identifying and eliminating outliers in the memory buffer by choosing samples from label-homogeneous subpopulations. We show that a space with a high homogeneity is related to a feature space that is more representative of the class distribution. In practice, MOE removes a sample if it is surrounded by samples from different labels. We demonstrate the effectiveness of MOE on CIFAR-10, CIFAR-100, and CORe50, outperforming previous well-known memory population methods.

翻译：灾难性遗忘——在学习新任务时遗忘先前所学任务的现象——是开发连续学习算法的主要障碍。缓解遗忘的一种常用方法是使用记忆缓冲区，该缓冲区存储先前学习任务的示例子集，以便在新任务训练中使用。填充记忆的默认方法是随机选择先前示例。然而，这一过程可能引入异常值或噪声样本，从而损害模型的泛化能力。本文提出了记忆异常消除（MOE）方法，通过从标签均匀的子群体中选择样本来识别并消除记忆缓冲区中的异常值。我们表明，高均匀性空间与更能代表类别分布的特征空间相关。在实际操作中，MOE会移除被不同标签样本包围的样本。我们在CIFAR-10、CIFAR-100和CORe50数据集上验证了MOE的有效性，其表现优于先前著名的记忆填充方法。