In large language model (LLM) unlearning, private information is required to be removed. Task arithmetic unlearns by subtracting a specific task vector (TV)--defined as the parameter difference between a privacy-information-tuned model and the original model. While efficient, it can cause over-forgetting by disrupting parameters essential for retaining other information. Motivated by the observation that each parameter exhibits different importance for forgetting versus retention, we propose a per-parameter task arithmetic (PerTA) mechanism to rescale the TV, allowing per-parameter adjustment. These weights quantify the relative importance of each parameter for forgetting versus retention, estimated via gradients (i.e., PerTA-grad) or the diagonal Fisher information approximation (i.e., PerTA-fisher). Moreover, we discuss the effectiveness of PerTA, extend it to a more general form, and provide further analysis. Extensive experiments demonstrate that PerTA consistently improves upon standard TV, and in many cases surpasses widely used training-based unlearning methods in both forgetting effectiveness and overall model utility. By retaining the efficiency of task arithmetic while mitigating over-forgetting, PerTA offers a principled and practical framework for LLM unlearning.

翻译：在大语言模型（LLM）的遗忘任务中，需要移除私有信息。任务算术遗忘法通过减去一个特定的任务向量（TV）来实现遗忘——该向量定义为经过隐私信息调优的模型与原始模型之间的参数差值。该方法虽然高效，但可能因扰动对保留其他信息至关重要的参数而导致过度遗忘。受每个参数对遗忘与保留具有不同重要性这一观察的启发，我们提出了一种基于参数的任务算术（PerTA）机制，通过重新缩放任务向量来实现逐参数调整。这些权重量化了每个参数对遗忘与保留的相对重要性，可通过梯度（即PerTA-grad）或对角费舍尔信息近似（即PerTA-fisher）进行估计。此外，我们讨论了PerTA的有效性，将其扩展为更一般的形式，并提供了进一步的分析。大量实验表明，PerTA在标准任务向量的基础上持续改进，并且在许多情况下，在遗忘效果和模型整体效用方面均超越了广泛使用的基于训练的遗忘方法。PerTA在保持任务算术效率的同时缓解了过度遗忘问题，为LLM遗忘提供了一个原则性且实用的框架。