Large language models internalize enormous parametric knowledge during pre-training. Concurrently, realistic applications necessitate external contextual knowledge to aid models on the underlying tasks. This raises a crucial dilemma known as knowledge conflicts, where the contextual knowledge clashes with the However, existing decoding works are specialized in resolving knowledge conflicts and could inadvertently deteriorate performance in absence of conflicts. In this paper, we propose an adaptive decoding method, termed as contextual information-entropy constraint decoding (COIECD), to discern whether the knowledge conflicts occur and resolve them. It can improve the model's faithfulness to conflicting context, and simultaneously maintain high performance among non- Our experiments show that COIECD exhibits strong performance and robustness over knowledge conflicts in realistic datasets. Code is available.

翻译：大型语言模型在预训练过程中内化了海量的参数化知识。与此同时，实际应用通常需要借助外部上下文知识来辅助模型完成底层任务。这引发了一个被称为“知识冲突”的关键困境，即上下文知识与模型内部参数知识发生矛盾。然而，现有的解码方法专门用于解决知识冲突，但在无冲突场景下可能无意中导致性能下降。本文提出一种自适应解码方法，称为上下文信息熵约束解码（COIECD），旨在识别知识冲突是否发生并予以解决。该方法能够提升模型对冲突上下文的忠实度，同时在非冲突场景下保持高性能。实验表明，COIECD在实际数据集中对知识冲突表现出强大的处理能力和鲁棒性。代码已开源。