Causal reasoning, the ability to identify cause-and-effect relationship, is crucial in human thinking. Although large language models (LLMs) succeed in many NLP tasks, it is still challenging for them to conduct complex causal reasoning like abductive reasoning and counterfactual reasoning. Given the fact that programming code may express causal relations more often and explicitly with conditional statements like ``if``, we want to explore whether Code-LLMs acquire better causal reasoning abilities. Our experiments show that compared to text-only LLMs, Code-LLMs with code prompts are significantly better in causal reasoning. We further intervene on the prompts from different aspects, and discover that the programming structure is crucial in code prompt design, while Code-LLMs are robust towards format perturbations.

翻译：因果推理，即识别因果关系的能力，在人类思维中至关重要。尽管大语言模型（LLMs）在许多自然语言处理任务中取得了成功，但它们在执行复杂因果推理（如溯因推理和反事实推理）时仍面临挑战。鉴于编程代码常通过条件语句（如“if”）更频繁且显式地表达因果关系，我们旨在探究代码大语言模型（Code-LLMs）是否具备更优的因果推理能力。实验表明，与纯文本LLMs相比，结合代码提示的Code-LLMs在因果推理方面表现显著提升。我们进一步从不同角度对提示进行干预，发现编程结构在代码提示设计中至关重要，而Code-LLMs对格式扰动具有稳健性。