Federated learning (FL) enables multiple clients to train a machine learning model collaboratively without exchanging their local data. Federated unlearning is an inverse FL process that aims to remove a specified target client's contribution in FL to satisfy the user's right to be forgotten. Most existing federated unlearning algorithms require the server to store the history of the parameter updates, which is not applicable in scenarios where the server storage resource is constrained. In this paper, we propose a simple-yet-effective subspace based federated unlearning method, dubbed SFU, that lets the global model perform gradient ascent in the orthogonal space of input gradient spaces formed by other clients to eliminate the target client's contribution without requiring additional storage. Specifically, the server first collects the gradients generated from the target client after performing gradient ascent, and the input representation matrix is computed locally by the remaining clients. We also design a differential privacy method to protect the privacy of the representation matrix. Then the server merges those representation matrices to get the input gradient subspace and updates the global model in the orthogonal subspace of the input gradient subspace to complete the forgetting task with minimal model performance degradation. Experiments on MNIST, CIFAR10, and CIFAR100 show that SFU outperforms several state-of-the-art (SOTA) federated unlearning algorithms by a large margin in various settings.

翻译：联邦学习（FL）允许多个客户端在不交换本地数据的情况下协作训练机器学习模型。联邦遗忘学习是FL的逆过程，旨在移除指定目标客户端在FL中的贡献，以满足用户被遗忘权的要求。现有的大多数联邦遗忘学习算法要求服务器存储参数更新的历史记录，这在服务器存储资源受限的场景中并不适用。本文提出一种简单而有效的基于子空间的联邦遗忘学习方法，称为SFU，该方法让全局模型在其他客户端形成的输入梯度空间的正交空间中进行梯度上升，以消除目标客户端的贡献，且无需额外存储。具体而言，服务器首先收集目标客户端在执行梯度上升后生成的梯度，而输入表示矩阵由剩余客户端本地计算得出。我们还设计了一种差分隐私方法来保护表示矩阵的隐私。然后，服务器合并这些表示矩阵以获取输入梯度子空间，并在输入梯度子空间的正交子空间中更新全局模型，从而以最小的模型性能损失完成遗忘任务。在MNIST、CIFAR10和CIFAR100上的实验表明，SFU在各种设置下均大幅优于目前最先进的（SOTA）联邦遗忘学习算法。