The parametric knowledge memorized by large language models (LLMs) becomes outdated quickly. In-context editing (ICE) is currently the most effective method for updating the knowledge of LLMs. Recent advancements involve enhancing ICE by modifying the decoding strategy, obviating the need for altering internal model structures or adjusting external prompts. However, this enhancement operates across the entire sequence generation, encompassing a plethora of non-critical tokens. In this work, we introduce $\textbf{A}$daptive $\textbf{T}$oken $\textbf{Bias}$er ($\textbf{ATBias}$), a new decoding technique designed to enhance ICE. It focuses on the tokens that are mostly related to knowledge during decoding, biasing their logits by matching key entities related to new and parametric knowledge. Experimental results show that ATBias significantly enhances ICE performance, achieving up to a 32.3% improvement over state-of-the-art ICE methods while incurring only half the latency. ATBias not only improves the knowledge editing capabilities of ICE but can also be widely applied to LLMs with negligible cost.

翻译：大型语言模型（LLM）所记忆的参数化知识会迅速过时。上下文编辑（ICE）是目前更新LLM知识最有效的方法。近期研究进展通过修改解码策略来增强ICE，从而无需改变模型内部结构或调整外部提示。然而，这种增强作用于整个序列生成过程，涉及大量非关键令牌。本文提出自适应令牌偏置器（ATBias），这是一种旨在增强ICE的新型解码技术。该方法在解码过程中聚焦于与知识最相关的令牌，通过匹配新旧参数化知识的关键实体来偏置其对数概率。实验结果表明，ATBias显著提升了ICE性能，相比最先进的ICE方法最高可获得32.3%的改进，同时仅产生一半的延迟。ATBias不仅增强了ICE的知识编辑能力，还能以可忽略的成本广泛应用于各类LLM。