Foundation models are commonly deployed as frozen feature extractors with a small trainable head to adapt to private, user-generated data in federated settings. The ``right to be forgotten'' requires removing the influence of specific samples or users from the trained model on demand. Existing federated unlearning methods target general deep models and rely on approximate reconstruction or selective retraining, making exactness costly or elusive. We study this problem in a practically relevant but under-explored regime: a frozen foundation model with a ridge-regression head. The exact optimum depends on the data only through two additive sufficient statistics, which we turn into a communication protocol supporting an arbitrary stream of add and delete requests via fixed-size messages. The server maintains a head that is, in exact arithmetic, pointwise identical to centralized retraining after every request. We provide deterministic retrain-equivalence guarantees, order and partition invariance, two server-side variants, and a Bayesian certificate of zero KL divergence. Experiments on four benchmarks confirm the guarantees: both variants match centralized ridge retraining to within $10^{-9}$ relative Frobenius error and complete each request at orders-of-magnitude lower cost than federated retraining baselines.
翻译:基础模型在联邦场景中通常作为冻结的特征提取器部署,并搭配一个小的可训练头以适应私有的用户生成数据。“被遗忘权”要求按需移除特定样本或用户对已训练模型的影响。现有联邦遗忘方法针对通用深度模型,依赖于近似重建或选择性重训练,使得精确性代价高昂或难以实现。我们在此研究一个实际相关但探索不足的场景:冻结基础模型与脊回归头。精确最优解仅通过两个加性充分统计量依赖于数据,我们将其转化为一种通信协议,支持通过固定大小消息处理任意流的添加和删除请求。服务器维护一个在精确算术意义上点对点与每次请求后的集中式重训练一致的头。我们提供确定性的重训练等价保证、顺序与划分不变性、两种服务器端变体,以及零KL散度的贝叶斯认证。在四个基准上的实验证实了这些保证:两种变体与集中式脊回归重训练的相对Frobenius误差均在$10^{-9}$以内,且每个请求的完成成本比联邦重训练基线低数个数量级。