Machine unlearning (MU) is essential for enforcing the right to be forgotten in machine learning systems. A key challenge of MU is how to reliably audit whether a model has truly forgotten specified training data. Membership Inference Attacks (MIAs) are widely used for unlearning auditing, where samples that evade membership detection are often regarded as successfully forgotten. After carefully revisiting the reliability of MIA, we show that this assumption is flawed: failed membership inference does not imply true forgetting. We theoretically demonstrate that MIA-based auditing, when formulated as a binary classification problem, inevitably incurs statistical errors whose magnitude cannot be observed during the auditing process. This leads to overly optimistic evaluations of unlearning performance, while incurring substantial computational overhead due to shadow model training. To address these limitations, we propose Statistical Membership Inference Attack (SMIA), a novel training-free and highly effective auditing framework. SMIA directly compares the distributions of member and non-member data using statistical tests, eliminating the need for learned attack models. Moreover, SMIA outputs both a forgetting rate and a corresponding confidence interval, enabling quantified reliability of the auditing results. Extensive experiments show that SMIA provides more reliable auditing with significantly lower computational cost than existing MIA-based approaches. Notably, the theoretical guarantees and empirical effectiveness of SMIA suggest it as a new paradigm for reliable machine unlearning auditing.

翻译：机器遗忘对于在机器学习系统中实现被遗忘权至关重要。机器遗忘的一个核心挑战是如何可靠地审计模型是否真正遗忘了指定的训练数据。成员推理攻击被广泛用于遗忘审计，其中逃避成员身份检测的样本通常被视为已成功遗忘。在仔细重新审视MIA的可靠性后，我们发现这一假设存在缺陷：成员推理失败并不意味着真正的遗忘。我们从理论上证明，当将基于MIA的审计表述为一个二元分类问题时，其不可避免地会产生统计误差，且该误差的大小在审计过程中无法被观测。这导致了对遗忘性能过于乐观的评估，同时由于影子模型的训练而产生了巨大的计算开销。为了解决这些局限性，我们提出了统计成员推理攻击，这是一种新颖的、无需训练且高效的审计框架。SMIA直接使用统计检验比较成员数据与非成员数据的分布，从而无需训练攻击模型。此外，SMIA同时输出遗忘率及其相应的置信区间，从而实现了审计结果的可量化可靠性。大量实验表明，与现有的基于MIA的方法相比，SMIA能以显著更低的计算成本提供更可靠的审计。值得注意的是，SMIA的理论保证和实证有效性表明，它可作为实现可靠机器遗忘审计的新范式。