Despite the notable advancements of existing prompting methods, such as In-Context Learning and Chain-of-Thought for Large Language Models (LLMs), they still face challenges related to various biases. Traditional debiasing methods primarily focus on the model training stage, including approaches based on data augmentation and reweighting, yet they struggle with the complex biases inherent in LLMs. To address such limitations, the causal relationship behind the prompting methods is uncovered using a structural causal model, and a novel causal prompting method based on front-door adjustment is proposed to effectively mitigate LLMs biases. In specific, causal intervention is achieved by designing the prompts without accessing the parameters and logits of LLMs. The chain-of-thought generated by LLM is employed as the mediator variable and the causal effect between input prompts and output answers is calculated through front-door adjustment to mitigate model biases. Moreover, to accurately represent the chain-of-thoughts and estimate the causal effects, contrastive learning is used to fine-tune the encoder of chain-of-thought by aligning its space with that of the LLM. Experimental results show that the proposed causal prompting approach achieves excellent performance across seven natural language processing datasets on both open-source and closed-source LLMs.

翻译：尽管现有提示方法（如大语言模型的上下文学习与思维链）已取得显著进展，其仍面临各类偏差的挑战。传统去偏方法主要集中于模型训练阶段，包括基于数据增强与重加权的方法，但难以应对大语言模型固有的复杂偏差。为突破此局限，本研究通过结构因果模型揭示提示方法背后的因果关系，并提出一种基于前门调整的新型因果提示方法，以有效缓解大语言模型的偏差。具体而言，该方法通过设计提示词实现因果干预，而无需访问大语言模型的参数与逻辑输出。我们将大语言模型生成的思维链作为中介变量，通过前门调整计算输入提示与输出答案间的因果效应以削弱模型偏差。此外，为精确表征思维链并估计因果效应，本研究采用对比学习对思维链编码器进行微调，使其表征空间与大语言模型空间对齐。实验结果表明，所提出的因果提示方法在开源与闭源大语言模型上，于七个自然语言处理数据集中均取得了优异性能。