Neural language models (LMs) have been extensively trained on vast corpora to store factual knowledge about various aspects of the world described in texts. Current technologies typically employ knowledge editing methods or specific prompts to modify LM outputs. However, existing knowledge editing methods are costly and inefficient, struggling to produce appropriate text. Additionally, prompt engineering is opaque and requires significant effort to find suitable prompts. To address these issues, we introduce a new method called PSPEM (Prefix Soft Prompt Editing Method), that can be used for a lifetime with just one training. It resolves the inefficiencies and generalizability issues in knowledge editing methods and overcomes the opacity of prompt engineering by automatically seeking optimal soft prompts. Specifically, PSPEM utilizes a prompt encoder and an encoding converter to refine key information in prompts and uses prompt alignment techniques to guide model generation, ensuring text consistency and adherence to the intended structure and content, thereby maintaining an optimal balance between efficiency and accuracy. We have validated the effectiveness of PSPEM through knowledge editing and attribute inserting. On the COUNTERFACT dataset, PSPEM achieved nearly 100\% editing accuracy and demonstrated the highest level of fluency. We further analyzed the similarities between PSPEM and original prompts and their impact on the model's internals. The results indicate that PSPEM can serve as an alternative to original prompts, supporting the model in effective editing.

翻译：神经语言模型（LM）在大规模文本语料上经过广泛训练，以存储关于文本描述世界各个方面的知识。现有技术通常采用知识编辑方法或特定提示来修改LM的输出。然而，现有的知识编辑方法成本高且效率低，难以生成恰当的文本。此外，提示工程具有不透明性，且需要大量努力才能找到合适的提示。为解决这些问题，我们提出了一种名为PSPEM（前缀软提示编辑方法）的新方法，该方法仅需一次训练即可终身使用。它解决了知识编辑方法中的低效性和泛化性问题，并通过自动寻找最优软提示克服了提示工程的不透明性。具体而言，PSPEM利用提示编码器和编码转换器来优化提示中的关键信息，并使用提示对齐技术指导模型生成，确保文本一致性以及遵循预期的结构和内容，从而在效率与准确性之间保持最佳平衡。我们通过知识编辑和属性插入验证了PSPEM的有效性。在COUNTERFACT数据集上，PSPEM实现了近乎100%的编辑准确率，并展现出最高水平的流畅性。我们进一步分析了PSPEM与原始提示之间的相似性及其对模型内部结构的影响。结果表明，PSPEM可作为原始提示的替代方案，支持模型进行有效的编辑。