Defeasibility in causal reasoning implies that the causal relationship between cause and effect can be strengthened or weakened. Namely, the causal strength between cause and effect should increase or decrease with the incorporation of strengthening arguments (supporters) or weakening arguments (defeaters), respectively. However, existing works ignore defeasibility in causal reasoning and fail to evaluate existing causal strength metrics in defeasible settings. In this work, we present $\delta$-CAUSAL, the first benchmark dataset for studying defeasibility in causal reasoning. $\delta$-CAUSAL includes around 11K events spanning ten domains, featuring defeasible causality pairs, i.e., cause-effect pairs accompanied by supporters and defeaters. We further show current causal strength metrics fail to reflect the change of causal strength with the incorporation of supporters or defeaters in $\delta$-CAUSAL. To this end, we propose CESAR (Causal Embedding aSsociation with Attention Rating), a metric that measures causal strength based on token-level causal relationships. CESAR achieves a significant 69.7% relative improvement over existing metrics, increasing from 47.2% to 80.1% in capturing the causal strength change brought by supporters and defeaters. We further demonstrate even Large Language Models (LLMs) like GPT-3.5 still lag 4.5 and 10.7 points behind humans in generating supporters and defeaters, emphasizing the challenge posed by $\delta$-CAUSAL.

翻译：因果推理中的可废止性意味着因果关系可被增强或削弱。具体而言，因果强度应随着增强论据（支持者）或削弱论据（废止者）的引入而相应增强或减弱。然而，现有研究忽视了因果推理中的可废止性，也未能评估现有因果强度度量在可废止场景下的表现。本工作提出了$\delta$-CAUSAL——首个用于研究因果推理可废止性的基准数据集。$\delta$-CAUSAL包含跨越十个领域的约1.1万个事件，其特点是包含可废止的因果关系对，即每个因果对都配有相应的支持者与废止者。我们进一步发现，现有因果强度度量方法在$\delta$-CAUSAL中无法有效反映支持者或废止者引入所带来的因果强度变化。为此，我们提出了CESAR（基于注意力评分的因果嵌入关联度量），这是一种基于词元级因果关系的因果强度度量方法。在捕捉支持者与废止者引起的因果强度变化方面，CESAR实现了69.7%的相对性能提升，准确率从47.2%提升至80.1%，显著优于现有度量方法。我们进一步证明，即使是GPT-3.5等大型语言模型，在生成支持者与废止者方面仍分别落后人类水平4.5和10.7个百分点，这凸显了$\delta$-CAUSAL所提出的挑战。