The reasoning abilities of Large Language Models (LLMs) are attracting increasing attention. In this work, we focus on causal reasoning and address the task of establishing causal relationships based on correlation information, a highly challenging problem on which several LLMs have shown poor performance. We introduce a prompting strategy for this problem that breaks the original task into fixed subquestions, with each subquestion corresponding to one step of a formal causal discovery algorithm, the PC algorithm. The proposed prompting strategy, PC-SubQ, guides the LLM to follow these algorithmic steps, by sequentially prompting it with one subquestion at a time, augmenting the next subquestion's prompt with the answer to the previous one(s). We evaluate our approach on an existing causal benchmark, Corr2Cause: our experiments indicate a performance improvement across five LLMs when comparing PC-SubQ to baseline prompting strategies. Results are robust to causal query perturbations, when modifying the variable names or paraphrasing the expressions.

翻译：大型语言模型（LLMs）的推理能力正受到日益广泛的关注。本研究聚焦于因果推理，致力于解决基于相关性信息建立因果关系这一极具挑战性的任务——已有研究表明多个LLM在此任务上表现欠佳。我们针对该问题提出一种提示策略，将原始任务分解为固定子问题，每个子问题对应形式化因果发现算法（PC算法）的一个步骤。所提出的提示策略PC-SubQ通过按序向LLM逐次提示单个子问题，并将前序子问题的答案融入后续提示中，从而引导LLM遵循这些算法步骤。我们在现有因果基准测试集Corr2Cause上评估了该方法：实验结果表明，相较于基线提示策略，PC-SubQ在五种LLM上均实现了性能提升。该结果对因果查询扰动具有稳健性，包括变量名修改和表达方式转述等情况。