Knowledge editing for large language models can offer an efficient solution to alter a model's behavior without negatively impacting the overall performance. However, the current approach encounters issues with limited generalizability across tasks, necessitating one distinct editor for each task, which significantly hinders the broader applications. To address this, we take the first step to analyze the multi-task generalization issue in knowledge editing. Specifically, we develop an instruction-based editing technique, termed InstructEdit, which facilitates the editor's adaptation to various task performances simultaneously using simple instructions. With only one unified editor for each LLM, we empirically demonstrate that InstructEdit can improve the editor's control, leading to an average 14.86% increase in Reliability in multi-task editing setting. Furthermore, experiments involving holdout unseen task illustrate that InstructEdit consistently surpass previous strong baselines. To further investigate the underlying mechanisms of instruction-based knowledge editing, we analyze the principal components of the editing gradient directions, which unveils that instructions can help control optimization direction with stronger OOD generalization. Code and datasets will be available in https://github.com/zjunlp/EasyEdit.

翻译：大语言模型的知识编辑能够在不影响整体性能的前提下，高效地改变模型行为。然而，当前方法在多任务泛化性上存在局限，需为每个任务配备独立的编辑器，这严重阻碍了其广泛应用。为解决此问题，我们率先分析了知识编辑中的多任务泛化问题。具体而言，我们提出了一种基于指令的编辑技术——InstructEdit，通过简单指令即可使编辑器同时适应多种任务性能。针对每个LLM仅使用一个统一编辑器，实验表明InstructEdit能提升编辑器的控制能力，在多任务编辑场景下平均可靠性提高14.86%。此外，涉及未见任务的留出实验显示，InstructEdit始终优于先前强基线方法。为进一步探究基于指令的知识编辑的内在机制，我们分析了编辑梯度方向的主成分，揭示了指令能够通过更强的分布外泛化能力控制优化方向。代码与数据集将开源至https://github.com/zjunlp/EasyEdit。