Recent regulation on right-to-be-forgotten emerges tons of interest in unlearning pre-trained machine learning models. While approximating a straightforward yet expensive approach of retrain-from-scratch, recent machine unlearning methods unlearn a sample by updating weights to remove its influence on the weight parameters. In this paper, we introduce a simple yet effective approach to remove a data influence on the deep generative model. Inspired by works in multi-task learning, we propose to manipulate gradients to regularize the interplay of influence among samples by projecting gradients onto the normal plane of the gradients to be retained. Our work is agnostic to statistics of the removal samples, outperforming existing baselines while providing theoretical analysis for the first time in unlearning a generative model.

翻译：近期关于“被遗忘权”的法规引发了大量对预训练机器学习模型进行遗忘研究的热点。尽管近似了一种简单但成本高昂的从头再训练方法，近期机器遗忘方法通过更新权重来消除样本对权重参数的影响，从而遗忘该样本。本文提出一种简单有效的方法，以消除数据对深度生成模型的影响。受多任务学习研究的启发，我们提出通过将梯度投影到待保留梯度的法平面上，来操控梯度从而规范样本间影响的相互作用。我们的方法对被遗忘样本的统计特性不敏感，在性能上超越现有基线方法，且首次为生成模型的遗忘问题提供了理论分析。