Machine unlearning, the ability for a machine learning model to forget, is becoming increasingly important to comply with data privacy regulations, as well as to remove harmful, manipulated, or outdated information. The key challenge lies in forgetting specific information while protecting model performance on the remaining data. While current state-of-the-art methods perform well, they typically require some level of retraining over the retained data, in order to protect or restore model performance. This adds computational overhead and mandates that the training data remain available and accessible, which may not be feasible. In contrast, other methods employ a retrain-free paradigm, however, these approaches are prohibitively computationally expensive and do not perform on par with their retrain-based counterparts. We present Selective Synaptic Dampening (SSD), a novel two-step, post hoc, retrain-free approach to machine unlearning which is fast, performant, and does not require long-term storage of the training data. First, SSD uses the Fisher information matrix of the training and forgetting data to select parameters that are disproportionately important to the forget set. Second, SSD induces forgetting by dampening these parameters proportional to their relative importance to the forget set with respect to the wider training data. We evaluate our method against several existing unlearning methods in a range of experiments using ResNet18 and Vision Transformer. Results show that the performance of SSD is competitive with retrain-based post hoc methods, demonstrating the viability of retrain-free post hoc unlearning approaches.

翻译：机器遗忘，即机器学习模型具备遗忘能力，正日益重要，以满足数据隐私法规要求，并移除有害、被篡改或过时信息。核心挑战在于遗忘特定信息的同时，保护模型在保留数据上的性能。当前最先进的方法虽表现良好，但通常需要对保留数据进行一定程度的重训练，以维持或恢复模型性能。这会增加计算开销，并要求训练数据始终可用且可访问，而这可能并不可行。相比之下，其他方法采用无需重训练的范式，但这些方法计算成本过高，且性能不如基于重训练的方法。我们提出选择性突触抑制（Selective Synaptic Dampening, SSD），一种新颖的两步事后无需重训练方法，该方法快速、高效，且无需长期存储训练数据。首先，SSD利用训练数据和遗忘数据的Fisher信息矩阵，选择对遗忘集重要性不成比例的参数。其次，SSD通过按这些参数相对于整个训练数据对遗忘集的相对重要性进行抑制，来诱导遗忘。我们使用ResNet18和Vision Transformer在系列实验中评估了该方法与现有多种遗忘方法的性能。结果表明，SSD的性能与基于重训练的事后方法具有竞争力，证明了无需重训练的事后遗忘方法的可行性。