Knowledge-intensive language understanding tasks require Language Models (LMs) to integrate relevant context, mitigating their inherent weaknesses, such as incomplete or outdated knowledge. Nevertheless, studies indicate that LMs often ignore the provided context as it can conflict with the pre-existing LM's memory learned during pre-training. Moreover, conflicting knowledge can already be present in the LM's parameters, termed intra-memory conflict. Existing works have studied the two types of knowledge conflicts only in isolation. We conjecture that the (degree of) intra-memory conflicts can in turn affect LM's handling of context-memory conflicts. To study this, we introduce the DYNAMICQA dataset, which includes facts with a temporal dynamic nature where a fact can change with a varying time frequency and disputable dynamic facts, which can change depending on the viewpoint. DYNAMICQA is the first to include real-world knowledge conflicts and provide context to study the link between the different types of knowledge conflicts. With the proposed dataset, we assess the use of uncertainty for measuring the intra-memory conflict and introduce a novel Coherent Persuasion (CP) score to evaluate the context's ability to sway LM's semantic output. Our extensive experiments reveal that static facts, which are unlikely to change, are more easily updated with additional context, relative to temporal and disputable facts.

翻译：知识密集型语言理解任务要求语言模型（LMs）整合相关上下文，以弥补其固有的弱点，例如知识不完整或过时。然而，研究表明，语言模型常常忽略所提供的上下文，因为它可能与预训练期间习得的、预先存在于模型记忆中的知识相冲突。此外，冲突性知识可能已经存在于语言模型的参数中，这被称为内部记忆冲突。现有研究仅孤立地探讨了这两种类型的知识冲突。我们推测，内部记忆冲突的（程度）反过来会影响语言模型处理上下文-记忆冲突的方式。为研究此问题，我们引入了DYNAMICQA数据集，该数据集包含具有时间动态性的事实（即事实可能以不同的时间频率发生变化）以及可争议的动态事实（即事实可能因视角不同而改变）。DYNAMICQA是首个包含现实世界知识冲突并提供上下文以研究不同类型知识冲突之间关联的数据集。利用所提出的数据集，我们评估了使用不确定性来度量内部记忆冲突的方法，并引入了一种新颖的连贯说服力（CP）分数，以评估上下文影响语言模型语义输出的能力。我们的大量实验表明，相对于时间和可争议的事实，不太可能发生变化的静态事实更容易通过额外上下文进行更新。