Machine unlearning is a prominent and challenging field, driven by regulatory demands for user data deletion and heightened privacy awareness. Existing approaches involve retraining model or multiple finetuning steps for each deletion request, often constrained by computational limits and restricted data access. In this work, we introduce a novel class unlearning algorithm designed to strategically eliminate specific classes from the learned model. Our algorithm first estimates the Retain and the Forget Spaces using Singular Value Decomposition on the layerwise activations for a small subset of samples from the retain and unlearn classes, respectively. We then compute the shared information between these spaces and remove it from the forget space to isolate class-discriminatory feature space. Finally, we obtain the unlearned model by updating the weights to suppress the class discriminatory features from the activation spaces. We demonstrate our algorithm's efficacy on ImageNet using a Vision Transformer with only $\sim 1.5\%$ drop in retain accuracy compared to the original model while maintaining under $1\%$ accuracy on the unlearned class samples. Further, our algorithm consistently performs well when subject to Membership Inference Attacks showing $7.8\%$ improvement on average across a variety of image classification datasets and network architectures, as compared to other baselines while being $\sim 6 \times$ more computationally efficient. Our code is available at https://github.com/sangamesh-kodge/class_forgetting.

翻译：机器遗忘是一个重要且富有挑战性的领域，其驱动力来自用户数据删除的法规要求以及日益增长的隐私意识。现有方法通常需要对每个删除请求重新训练模型或执行多次微调步骤，但往往受限于计算能力和有限的数据访问权限。本文提出一种新颖的类别遗忘算法，旨在从已学习模型中战略性地移除特定类别。该算法首先对保留类别和遗忘类别的一小部分样本的逐层激活进行奇异值分解，分别估计保留空间和遗忘空间；随后计算这两个空间之间的共享信息，并从遗忘空间中移除该信息，以隔离类别判别性特征空间；最后，通过更新权重以抑制激活空间中的类别判别性特征，获得遗忘后的模型。我们在ImageNet数据集上使用Vision Transformer验证了该算法的有效性，仅需保持原模型保留准确率下降约1.5%，同时确保遗忘类别样本的准确率低于1%。此外，该算法在成员推理攻击下表现稳定，相较于其他基线方法，在多种图像分类数据集和网络架构上平均改进7.8%，同时计算效率提升约6倍。我们的代码已开源：https://github.com/sangamesh-kodge/class_forgetting。