Recent data regulations necessitate machine unlearning (MU): The removal of the effect of specific examples from the model. While exact unlearning is possible by conducting a model retraining with the remaining data from scratch, its computational cost has led to the development of approximate but efficient unlearning schemes. Beyond data-centric MU solutions, we advance MU through a novel model-based viewpoint: sparsification via weight pruning. Our results in both theory and practice indicate that model sparsity can boost the multi-criteria unlearning performance of an approximate unlearner, closing the approximation gap, while continuing to be efficient. With this insight, we develop two new sparsity-aware unlearning meta-schemes, termed `prune first, then unlearn' and `sparsity-aware unlearning'. Extensive experiments show that our findings and proposals consistently benefit MU in various scenarios, including class-wise data scrubbing, random data scrubbing, and backdoor data forgetting. One highlight is the 77% unlearning efficacy gain of fine-tuning (one of the simplest approximate unlearning methods) in the proposed sparsity-aware unlearning paradigm. Codes are available at https://github.com/OPTML-Group/Unlearn-Sparse.

翻译：近期数据法规要求实现机器反学习（MU）：从模型中移除特定样本的影响。尽管通过使用剩余数据从头重新训练模型可以实现精确反学习，但其计算成本催生了近似但高效的反学习方案。不同于以数据为中心的MU解决方案，我们通过全新的基于模型视角推进MU：基于权重剪枝的稀疏化方法。我们的理论与实践经验均表明，模型稀疏性能够提升近似反学习器的多准则反学习性能，在保持高效的同时缩小近似差距。基于这一发现，我们开发了两种新的稀疏感知反学习元方案，称为"先剪枝后反学习"与"稀疏感知反学习"。大量实验表明，我们的发现与方案在各类场景（包括按类数据擦除、随机数据擦除和后门数据遗忘）中均能持续提升MU性能。一个突出亮点是：在提出的稀疏感知反学习范式下，微调（最简单的近似反学习方法之一）的反学习效能提升了77%。代码已开源至 https://github.com/OPTML-Group/Unlearn-Sparse。