Machine unlearning, the process of selectively removing data from trained models, is increasingly crucial for addressing privacy concerns and knowledge gaps post-deployment. Despite this importance, existing approaches are often heuristic and lack formal guarantees. In this paper, we analyze the fundamental utility, time, and space complexity trade-offs of approximate unlearning, providing rigorous certification analogous to differential privacy. For in-distribution forget data -- data similar to the retain set -- we show that a surprisingly simple and general procedure, empirical risk minimization with output perturbation, achieves tight unlearning-utility-complexity trade-offs, addressing a previous theoretical gap on the separation from unlearning "for free" via differential privacy, which inherently facilitates the removal of such data. However, such techniques fail with out-of-distribution forget data -- data significantly different from the retain set -- where unlearning time complexity can exceed that of retraining, even for a single sample. To address this, we propose a new robust and noisy gradient descent variant that provably amortizes unlearning time complexity without compromising utility.

翻译：机器遗忘，即从训练模型中选择性移除数据的过程，对于解决部署后的隐私关切和知识缺口日益重要。尽管其重要性显著，现有方法通常具有启发性且缺乏形式化保证。本文分析了近似遗忘的基本效用、时间与空间复杂性权衡，提供了类似于差分隐私的严格认证。对于分布内遗忘数据——即与保留集相似的数据——我们证明了一种出奇简单且通用的程序，即结合输出扰动的经验风险最小化，能够实现紧致的遗忘-效用-复杂性权衡，从而弥补了先前关于其与通过差分隐私实现的“免费”遗忘之间存在分离的理论空白，而差分隐私本质上便于移除此类数据。然而，此类技术在处理分布外遗忘数据——即与保留集显著不同的数据——时会失效，此时即使对于单个样本，遗忘的时间复杂性也可能超过重新训练。为解决此问题，我们提出了一种新的鲁棒且带噪声的梯度下降变体，该变体可证明地分摊遗忘时间复杂性，同时不损害效用。