By providing external information to large language models (LLMs), tool augmentation (including retrieval augmentation) has emerged as a promising solution for addressing the limitations of LLMs' static parametric memory. However, how receptive are LLMs to such external evidence, especially when the evidence conflicts with their parametric memory? We present the first comprehensive and controlled investigation into the behavior of LLMs when encountering knowledge conflicts. We propose a systematic framework to elicit high-quality parametric memory from LLMs and construct the corresponding counter-memory, which enables us to conduct a series of controlled experiments. Our investigation reveals seemingly contradicting behaviors of LLMs. On the one hand, different from prior wisdom, we find that LLMs can be highly receptive to external evidence even when that conflicts with their parametric memory, given that the external evidence is coherent and convincing. On the other hand, LLMs also demonstrate a strong confirmation bias when the external evidence contains some information that is consistent with their parametric memory, despite being presented with conflicting evidence at the same time. These results pose important implications that are worth careful consideration for the further development and deployment of tool- and retrieval-augmented LLMs. Resources are available at https://github.com/OSU-NLP-Group/LLM-Knowledge-Conflict.

翻译：通过向大语言模型（LLMs）提供外部信息，工具增强（包括检索增强）已成为解决LLMs静态参数记忆局限性的有前景方案。然而，当外部证据与参数记忆相冲突时，LLMs对此类证据的接受程度如何？我们首次对LLMs遭遇知识冲突时的行为进行了全面且受控的研究。我们提出系统性框架来引导LLMs生成高质量参数记忆，并构建相应的反记忆，从而开展一系列受控实验。研究揭示了LLMs看似矛盾的行为：一方面，与先前认知不同，我们发现当外部证据连贯且令人信服时，即便与参数记忆相冲突，LLMs也能高度接受外部证据；另一方面，当外部证据包含与参数记忆一致的信息时，即便同时存在冲突证据，LLMs仍表现出强烈的确认偏误。这些发现对工具增强与检索增强LLMs的进一步开发与部署具有重要且需审慎考量的启示。资源详见https://github.com/OSU-NLP-Group/LLM-Knowledge-Conflict。