In response to recent data regulation requirements, machine unlearning (MU) has emerged as a critical process to remove the influence of specific examples from a given model. Although exact unlearning can be achieved through complete model retraining using the remaining dataset, the associated computational costs have driven the development of efficient, approximate unlearning techniques. Moving beyond data-centric MU approaches, our study introduces a novel model-based perspective: model sparsification via weight pruning, which is capable of reducing the gap between exact unlearning and approximate unlearning. We show in both theory and practice that model sparsity can boost the multi-criteria unlearning performance of an approximate unlearner, closing the approximation gap, while continuing to be efficient. This leads to a new MU paradigm, termed prune first, then unlearn, which infuses a sparse model prior into the unlearning process. Building on this insight, we also develop a sparsity-aware unlearning method that utilizes sparsity regularization to enhance the training process of approximate unlearning. Extensive experiments show that our proposals consistently benefit MU in various unlearning scenarios. A notable highlight is the 77% unlearning efficacy gain of fine-tuning (one of the simplest unlearning methods) when using sparsity-aware unlearning. Furthermore, we demonstrate the practical impact of our proposed MU methods in addressing other machine learning challenges, such as defending against backdoor attacks and enhancing transfer learning. Codes are available at https://github.com/OPTML-Group/Unlearn-Sparse.

翻译：针对近期数据法规的要求，机器遗忘（MU）已成为从给定模型中移除特定样本影响的关键过程。尽管完全重训练（使用剩余数据集）可实现精确遗忘，但高计算成本催生了高效近似遗忘方法的发展。本研究超越以数据为中心的MU方法，提出一种新颖的基于模型的视角：通过权重剪枝实现模型稀疏化，该方法能够缩小精确遗忘与近似遗忘之间的差距。我们从理论与实践中证明，模型稀疏性可提升近似遗忘算法的多准则遗忘性能，缩小近似误差，同时保持高效性。由此衍生出新的MU范式——"先剪枝后遗忘"，将稀疏模型先验融入遗忘过程。基于此洞察，我们进一步开发了稀疏感知遗忘方法，利用稀疏正则化增强近似遗忘的训练过程。大量实验表明，我们的方案能在多种遗忘场景中持续提升MU性能。值得关注的是，采用稀疏感知遗忘后，微调（最简单的遗忘方法之一）的遗忘效能提升了77%。此外，我们展示了所提MU方法在应对后门攻击防御、迁移学习增强等机器学习挑战中的实际效果。代码已开源至https://github.com/OPTML-Group/Unlearn-Sparse。