Advances in model editing through neuron pruning hold promise for removing undesirable concepts from large language models. However, it remains unclear whether models have the capacity to reacquire pruned concepts after editing. To investigate this, we evaluate concept relearning in models by tracking concept saliency and similarity in pruned neurons during retraining. Our findings reveal that models can quickly regain performance post-pruning by relocating advanced concepts to earlier layers and reallocating pruned concepts to primed neurons with similar semantics. This demonstrates that models exhibit polysemantic capacities and can blend old and new concepts in individual neurons. While neuron pruning provides interpretability into model concepts, our results highlight the challenges of permanent concept removal for improved model \textit{safety}. Monitoring concept reemergence and developing techniques to mitigate relearning of unsafe concepts will be important directions for more robust model editing. Overall, our work strongly demonstrates the resilience and fluidity of concept representations in LLMs post concept removal.

翻译：通过神经元剪枝进行模型编辑的进展为从大型语言模型中移除不良概念带来了希望。然而，模型在编辑后是否具备重新获取被剪枝概念的能力仍不明确。为探究这一问题，我们通过跟踪再训练过程中剪枝神经元内概念显著性及相似性，评估模型对概念的重学习情况。研究发现，模型在剪枝后能通过将高级概念迁移至较早层、并将被剪枝概念重新分配至具有相似语义的预激活神经元，从而快速恢复性能。这表明模型具备多语义能力，并能在单个神经元中融合新旧概念。尽管神经元剪枝为模型概念提供了可解释性，但我们的结果凸显了为提升模型安全性而永久移除概念所面临的挑战。监测概念复现并开发缓解不安全概念重学习的技术，将是实现更稳健模型编辑的重要方向。总体而言，本研究有力证明了大型语言模型在概念移除后概念表征的韧性与流动性。