Recent knowledge editing methods have primarily focused on modifying structured knowledge in large language models, heavily relying on the assumption that structured knowledge is stored as key-value pairs locally in MLP layers or specific neurons. However, this task setting overlooks the fact that a significant portion of real-world knowledge is stored in an unstructured format, characterized by long-form content, noise, and a complex yet comprehensive nature. The "knowledge locating" and "term-driven optimization" techniques conducted from the assumption used in previous methods (e.g., MEMIT) are ill-suited for unstructured knowledge. To address these challenges, we propose a novel unstructured knowledge editing method, namely UnKE, which extends previous assumptions in the layer dimension and token dimension. Firstly, in the layer dimension, we discard the "knowledge locating" step and treat first few layers as the key, which expand knowledge storage through layers to break the "knowledge stored locally" assumption. Next, we replace "term-driven optimization" with "cause-driven optimization" across all inputted tokens in the token dimension, directly optimizing the last layer of the key generator to perform editing to generate the required key vectors. By utilizing key-value pairs at the layer level, UnKE effectively represents and edits complex and comprehensive unstructured knowledge, leveraging the potential of both the MLP and attention layers. Results on newly proposed unstructure knowledge editing dataset (UnKEBench) and traditional structured datasets demonstrate that UnKE achieves remarkable performance, surpassing strong baselines.

翻译：近期知识编辑方法主要集中于修改大语言模型中的结构化知识，其核心假设是结构化知识以键值对形式局部存储在MLP层或特定神经元中。然而，这种任务设定忽略了一个事实：现实世界中相当比例的知识以非结构化形式存储，其特征表现为长文本内容、噪声干扰以及复杂而综合的本质。基于先前方法（如MEMIT）假设所采用的“知识定位”与“术语驱动优化”技术，并不适用于非结构化知识。为应对这些挑战，我们提出了一种新颖的非结构化知识编辑方法UnKE，该方法在层维度和令牌维度上扩展了先前的假设。首先，在层维度上，我们摒弃“知识定位”步骤，将前若干层视为键，通过层间扩展知识存储以打破“知识局部存储”的假设。其次，在令牌维度上，我们将“术语驱动优化”替换为覆盖所有输入令牌的“因果驱动优化”，直接优化键生成器的最后一层以执行编辑，从而生成所需的键向量。通过利用层级的键值对，UnKE充分发挥MLP层与注意力层的潜力，有效表征并编辑复杂综合的非结构化知识。在新提出的非结构化知识编辑数据集（UnKEBench）与传统结构化数据集上的实验结果表明，UnKE取得了显著性能提升，超越了现有强基线方法。