Machine unlearning (MU) for large language models has become critical for AI safety, yet existing methods fail to generalize to Mixture-of-Experts (MoE) architectures. We identify that traditional unlearning methods exploit MoE's architectural vulnerability: they manipulate routers to redirect queries away from knowledgeable experts rather than erasing knowledge, causing a loss of model utility and superficial forgetting. We propose Geometric Routing Invariance Preservation (GRIP), an algorithm-agnostic framework for unlearning for MoE. Our core contribution is a geometric constraint, implemented by projecting router gradient updates into an expert-specific null-space. Crucially, this decouples routing stability from parameter rigidity: while discrete expert selections remain stable for retained knowledge, the continuous router parameters remain plastic within the null space, allowing the model to undergo necessary internal reconfiguration to satisfy unlearning objectives. This forces the unlearning optimization to erase knowledge directly from expert parameters rather than exploiting the superficial router manipulation shortcut. GRIP functions as an adapter, constraining router parameter updates without modifying the underlying unlearning algorithm. Extensive experiments on large-scale MoE models demonstrate that our adapter eliminates expert selection shift (achieving over 95% routing stability) across all tested unlearning methods while preserving their utility. By preventing existing algorithms from exploiting MoE model's router vulnerability, GRIP adapts existing unlearning research from dense architectures to MoEs.

翻译：机器遗忘对于大型语言模型的安全性至关重要，然而现有方法无法推广至专家混合架构。我们发现传统遗忘方法利用了MoE的架构脆弱性：它们通过操纵路由器将查询从知识渊博的专家处转移，而非真正擦除知识，导致模型效用损失和表面性遗忘。本文提出几何路由不变性保持框架，这是一种面向MoE的算法无关遗忘框架。我们的核心贡献是一种几何约束，通过将路由器梯度更新投影至专家特定的零空间实现。关键之处在于，该设计将路由稳定性与参数刚性解耦：虽然离散的专家选择对保留知识保持稳定，但连续的路由器参数在零空间内仍保持可塑性，使模型能够进行必要的内部重构以满足遗忘目标。这迫使遗忘优化过程直接从专家参数中擦除知识，而非利用表面化的路由器操纵捷径。GRIP作为适配器运行，在不修改底层遗忘算法的前提下约束路由器参数更新。在大规模MoE模型上的大量实验表明，我们的适配器在所有测试的遗忘方法中消除了专家选择偏移（实现超过95%的路由稳定性），同时保持其效用。通过防止现有算法利用MoE模型的路由器脆弱性，GRIP将稠密架构的现有遗忘研究成功适配至MoE架构。