Machine unlearning strives to uphold the data owners' right to be forgotten by enabling models to selectively forget specific data. Recent methods suggest that one approach of data forgetting is by precomputing and storing statistics carrying second-order information to improve computational and memory efficiency. However, they rely on restrictive assumptions and the computation/storage suffer from the curse of model parameter dimensionality, making it challenging to apply to most deep neural networks. In this work, we propose a Hessian-free online unlearning method. We propose to maintain a statistical vector for each data point, computed through affine stochastic recursion approximation of the difference between retrained and learned models. Our proposed algorithm achieves near-instantaneous online unlearning as it only requires a vector addition operation. Based on the strategy that recollecting statistics for forgetting data, the proposed method significantly reduces the unlearning runtime. Experimental studies demonstrate that the proposed scheme surpasses existing results by orders of magnitude in terms of time and memory costs, while also enhancing accuracy.

翻译：机器遗忘旨在通过使模型能够选择性遗忘特定数据，维护数据所有者的被遗忘权。近期方法表明，一种数据遗忘方式是通过预计算并存储携带二阶信息的统计量，以提高计算和内存效率。然而，这些方法依赖于严格假设，且其计算/存储受到模型参数维度灾难的影响，难以应用于大多数深度神经网络。本文提出一种无黑塞矩阵的在线遗忘方法。我们建议为每个数据点维护一个统计向量，该向量通过仿射随机递归近似重训练模型与已学习模型之间的差异计算得到。所提算法仅需一次向量加法操作，即可实现近乎瞬间的在线遗忘。基于为遗忘数据重新收集统计量的策略，该方法显著缩短了遗忘运行时间。实验研究表明，所提方案在时间和内存成本上比现有方法提升数个数量级，同时提高了准确性。