Event temporal reasoning aims at identifying the temporal relations between two or more events from narratives. However, knowledge conflicts arise when there is a mismatch between the actual temporal relations of events in the context and the prior knowledge or biases learned by the model. In this paper, we propose to detect knowledge-conflict examples in event temporal reasoning using bias indicators, which include event relation prior bias, tense bias, narrative bias, and dependency bias. We define conflict examples as those where event relations are opposite to biased or prior relations. To mitigate event-related knowledge conflicts, we introduce a Counterfactual Data Augmentation (CDA) based method that can be applied to both Pre-trained Language Models (PLMs) and Large Language Models (LLMs) either as additional training data or demonstrations for In-Context Learning. Experiments suggest both PLMs and LLMs suffer from knowledge conflicts in event temporal reasoning, and CDA has the potential for reducing hallucination and improving model performance.
翻译:事件时间推理旨在从叙述中识别两个或多个事件之间的时间关系。然而,当上下文中事件的实际时间关系与模型习得的先验知识或偏差不匹配时,就会产生知识冲突。本文提出利用偏差指标(包括事件关系先验偏差、时态偏差、叙事偏差和依存偏差)来检测事件时间推理中的知识冲突实例。我们将冲突实例定义为事件关系与偏差或先验关系相反的情况。为了缓解事件相关的知识冲突,我们引入了一种基于反事实数据增强(CDA)的方法,该方法可作为额外的训练数据或上下文学习的示例,应用于预训练语言模型(PLMs)和大语言模型(LLMs)。实验表明,预训练语言模型和大语言模型在事件时间推理中都存在知识冲突,而反事实数据增强(CDA)具有减少幻觉和提升模型性能的潜力。