Neural language models (LMs) have been extensively trained on vast corpora to store factual knowledge about various aspects of the world described in texts. Current technologies typically employ knowledge editing methods or specific prompts to modify LM outputs. However, existing knowledge editing methods are costly and inefficient, struggling to produce appropriate text. Additionally, prompt engineering is opaque and requires significant effort to find suitable prompts. To address these issues, we introduce a new method called PSPEM (Prefix Soft Prompt Editing Method), that can be used for a lifetime with just one training. It resolves the inefficiencies and generalizability issues in knowledge editing methods and overcomes the opacity of prompt engineering by automatically seeking optimal soft prompts. Specifically, PSPEM utilizes a prompt encoder and an encoding converter to refine key information in prompts and uses prompt alignment techniques to guide model generation, ensuring text consistency and adherence to the intended structure and content, thereby maintaining an optimal balance between efficiency and accuracy. We have validated the effectiveness of PSPEM through knowledge editing and attribute inserting. On the COUNTERFACT dataset, PSPEM achieved nearly 100\% editing accuracy and demonstrated the highest level of fluency. We further analyzed the similarities between PSPEM and original prompts and their impact on the model's internals. The results indicate that PSPEM can serve as an alternative to original prompts, supporting the model in effective editing.

翻译：神经语言模型（LMs）已在大规模语料库上经过广泛训练，以存储文本所描述的世界多方面事实知识。当前技术通常采用知识编辑方法或特定提示来修改语言模型的输出。然而，现有知识编辑方法成本高昂且效率低下，难以生成合适的文本。此外，提示工程具有不透明性，需要耗费大量精力才能找到合适的提示。为解决这些问题，我们提出了一种名为PSPEM（前缀软提示编辑方法）的新方法，该方法仅需一次训练即可终身使用。它解决了知识编辑方法效率低和泛化能力不足的问题，并通过自动寻找最优软提示克服了提示工程的不透明性。具体而言，PSPEM利用提示编码器和编码转换器来精炼提示中的关键信息，并采用提示对齐技术指导模型生成，确保文本一致性以及遵循预期的结构和内容，从而在效率与准确性之间保持最佳平衡。我们通过知识编辑和属性插入验证了PSPEM的有效性。在COUNTERFACT数据集上，PSPEM实现了近100%的编辑准确率，并展现出最高的流畅度。我们进一步分析了PSPEM与原始提示之间的相似性及其对模型内部的影响。结果表明，PSPEM可作为原始提示的替代方案，支持模型进行有效编辑。