Federated Unlearning (FU) aims to delete specific training data from an ML model trained using Federated Learning (FL). We introduce QuickDrop, an efficient and original FU method that utilizes dataset distillation (DD) to accelerate unlearning and drastically reduces computational overhead compared to existing approaches. In QuickDrop, each client uses DD to generate a compact dataset representative of the original training dataset, called a distilled dataset, and uses this compact dataset during unlearning. To unlearn specific knowledge from the global model, QuickDrop has clients execute Stochastic Gradient Ascent with samples from the distilled datasets, thus significantly reducing computational overhead compared to conventional FU methods. We further increase the efficiency of QuickDrop by ingeniously integrating DD into the FL training process. By reusing the gradient updates produced during FL training for DD, the overhead of creating distilled datasets becomes close to negligible. Evaluations on three standard datasets show that, with comparable accuracy guarantees, QuickDrop reduces the duration of unlearning by 463.8x compared to model retraining from scratch and 65.1x compared to existing FU approaches. We also demonstrate the scalability of QuickDrop with 100 clients and show its effectiveness while handling multiple unlearning operations.

翻译：联邦遗忘学习旨在从联邦学习训练出的机器学习模型中删除特定训练数据。我们提出QuickDrop，这是一种高效且新颖的联邦遗忘学习方法，利用数据集蒸馏技术加速遗忘过程，相比现有方法大幅降低计算开销。在QuickDrop中，每个客户端使用数据集蒸馏生成一个代表原始训练数据的紧凑型数据集（称为蒸馏数据集），并在遗忘过程中使用该紧凑数据集。为了让全局模型遗忘特定知识，QuickDrop让客户端使用蒸馏数据集中的样本执行随机梯度上升，从而显著降低相较于传统联邦遗忘方法的计算开销。我们通过巧妙地将数据集蒸馏集成到联邦学习训练过程中，进一步提升了QuickDrop的效率。通过复用联邦学习训练过程中产生的梯度更新进行数据集蒸馏，创建蒸馏数据集的开销几乎可以忽略不计。在三个标准数据集上的评估表明，在精度保证相当的情况下，QuickDrop将遗忘时间相比从头重新训练模型缩短了463.8倍，相比现有联邦遗忘方法缩短了65.1倍。我们还展示了QuickDrop在100个客户端下的可扩展性，并证明了其在处理多次遗忘操作时的有效性。