Unlearning the data observed during the training of a machine learning (ML) model is an important task that can play a pivotal role in fortifying the privacy and security of ML-based applications. This paper raises the following questions: (i) can we unlearn a single or multiple class(es) of data from a ML model without looking at the full training data even once? (ii) can we make the process of unlearning fast and scalable to large datasets, and generalize it to different deep networks? We introduce a novel machine unlearning framework with error-maximizing noise generation and impair-repair based weight manipulation that offers an efficient solution to the above questions. An error-maximizing noise matrix is learned for the class to be unlearned using the original model. The noise matrix is used to manipulate the model weights to unlearn the targeted class of data. We introduce impair and repair steps for a controlled manipulation of the network weights. In the impair step, the noise matrix along with a very high learning rate is used to induce sharp unlearning in the model. Thereafter, the repair step is used to regain the overall performance. With very few update steps, we show excellent unlearning while substantially retaining the overall model accuracy. Unlearning multiple classes requires a similar number of update steps as for a single class, making our approach scalable to large problems. Our method is quite efficient in comparison to the existing methods, works for multi-class unlearning, does not put any constraints on the original optimization mechanism or network design, and works well in both small and large-scale vision tasks. This work is an important step towards fast and easy implementation of unlearning in deep networks. Source code: https://github.com/vikram2000b/Fast-Machine-Unlearning

翻译：在机器学习模型训练过程中遗忘所观测到的数据是一项重要任务，它能在强化基于机器学习的应用的隐私与安全性方面发挥关键作用。本文提出以下问题：(i) 能否在不完整查看训练数据的情况下，从模型中遗忘单个或多个类别的数据？(ii) 能否使遗忘过程快速且可扩展至大数据集，并推广至不同的深度网络？我们提出了一种新颖的机器遗忘框架，通过生成误差最大化噪声和基于损伤-修复的权重操控方法，为上述问题提供了高效解决方案。针对待遗忘类别，利用原始模型学习一个误差最大化噪声矩阵，并借助该矩阵操控模型权重以遗忘目标类别数据。我们引入损伤与修复步骤实现对网络权重的受控调整：在损伤步骤中，使用噪声矩阵配合极高学习率诱导模型发生剧烈遗忘；随后在修复步骤中恢复整体性能。通过极少的更新步数，我们实现了出色的遗忘效果，同时大幅保留了模型整体精度。遗忘多个类别所需的更新步数与遗忘单个类别相当，使我们的方法具备可扩展性以应对大规模问题。与现有方法相比，我们的方法更为高效，支持多类别遗忘，对原始优化机制或网络设计无任何约束，并在小规模及大规模视觉任务中均表现优异。这项工作为在深度网络中快速便捷实现遗忘迈出了重要一步。源代码：https://github.com/vikram2000b/Fast-Machine-Unlearning