Despite the significant achievements of existing prompting methods such as in-context learning and chain-of-thought for large language models (LLMs), they still face challenges of various biases. Traditional debiasing methods primarily focus on the model training stage, including data augmentation-based and reweight-based approaches, with the limitations of addressing the complex biases of LLMs. To address such limitations, the causal relationship behind the prompting methods is uncovered using a structural causal model, and a novel causal prompting method based on front-door adjustment is proposed to effectively mitigate the bias of LLMs. In specific, causal intervention is implemented by designing the prompts without accessing the parameters and logits of LLMs.The chain-of-thoughts generated by LLMs are employed as the mediator variable and the causal effect between the input prompt and the output answers is calculated through front-door adjustment to mitigate model biases. Moreover, to obtain the representation of the samples precisely and estimate the causal effect more accurately, contrastive learning is used to fine-tune the encoder of the samples by aligning the space of the encoder with the LLM. Experimental results show that the proposed causal prompting approach achieves excellent performance on 3 natural language processing datasets on both open-source and closed-source LLMs.

翻译：尽管现有的大语言模型提示方法（如情境学习和思维链）已取得显著成就，但仍面临各类偏差的挑战。传统去偏方法主要聚焦于模型训练阶段，包括基于数据增强和基于重加权的方法，但难以处理大语言模型的复杂偏差。为解决这一局限，本研究通过结构因果模型揭示提示方法背后的因果关系，提出一种基于前门调整的新型因果提示方法，有效缓解大语言模型的偏差。具体而言，通过设计无需访问大语言模型参数和逻辑值的提示实现因果干预，将大语言模型生成的思维链作为中介变量，并基于前门调整计算输入提示与输出答案之间的因果效应以减轻模型偏差。此外，为精确获取样本表示并更准确估计因果效应，采用对比学习通过对齐编码器空间与大语言模型对样本编码器进行微调。实验结果表明，所提出的因果提示方法在3个自然语言处理数据集上，无论是开源自闭源大语言模型均展现出优异性能。