Machine unlearning, the process of selectively removing data from trained models, is increasingly crucial for addressing privacy concerns and knowledge gaps post-deployment. Despite this importance, existing approaches are often heuristic and lack formal guarantees. In this paper, we analyze the fundamental utility, time, and space complexity trade-offs of approximate unlearning, providing rigorous certification analogous to differential privacy. For in-distribution forget data -- data similar to the retain set -- we show that a surprisingly simple and general procedure, empirical risk minimization with output perturbation, achieves tight unlearning-utility-complexity trade-offs, addressing a previous theoretical gap on the separation from unlearning "for free" via differential privacy, which inherently facilitates the removal of such data. However, such techniques fail with out-of-distribution forget data -- data significantly different from the retain set -- where unlearning time complexity can exceed that of retraining, even for a single sample. To address this, we propose a new robust and noisy gradient descent variant that provably amortizes unlearning time complexity without compromising utility.

翻译：机器遗忘——选择性移除已训练模型中的数据——对于解决部署后的隐私顾虑与知识空白日益关键。尽管其重要性，现有方法常基于启发式且缺乏形式化保证。本文分析了近似遗忘的基本效用、时间与空间复杂性权衡，提供了类似于差分隐私的严格认证。对于分布内遗忘数据（与保留集相似的数据），我们发现一种惊人简单且通用的程序——带输出扰动的经验风险最小化——实现了紧致的遗忘-效用-复杂性权衡，填补了此前关于通过差分隐私实现"免费遗忘"（其本质上便于移除此类数据）的理论分离空白。然而，这类技术无法处理分布外遗忘数据（与保留集显著不同的数据），此时即使针对单个样本，遗忘的时间复杂度也可能超过重新训练。为此，我们提出一种新型鲁棒噪声梯度下降变体，可在不损害效用的前提下可测地摊平遗忘时间复杂度。